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Abstract 

The  transition  of  technologies  ensures  that  our  warfighters  have  the  necessary  capabilities 
to  fight  the  ever  increasing  and  changing  global  threats.  Through  the  investigative  research  of 
this  study,  technology  transition  best  practices  were  taken  from  Department  of  Defense  (DOD) 
publications,  Air  Force  Instructions  (AFIs),  and  industry.  The  purpose  of  this  study  was  to  gain 
an  understating  on  what  extent  the  Air  Force  Research  Laboratories  were  implementing  known 
technology  transition  best  practices.  In  examining  the  use  of  technology  transition  best  practices, 
the  research  provides  insight  into  how  technologies  are  transitioned  from  AFRL  to  the 
acquisition  community.  This  study  relies  on  the  perspectives  and  knowledge  of  program 
managers  within  AFRL.  Data  analysis  along  with  an  extensive  literature  review  led  to 
recommendations  such  as:  an  implementation  of  a  technology  transition  team,  co-locating  AFRL 
program  managers  in  program  offices;  and  program  management  training  for  managers  in  the 
S&T  community.  It  was  revealed  that  only  some  aspects  of  technology  transition  best  practices 
were  being  implemented.  It  was  also  discovered  that  technology  transition  involves  multiple 
organizations  for  technologies  to  be  transitioned.  Based  on  the  data  collected,  it  was  found  that 
the  program  manager  needs  multiple  resources  and  the  right  knowledge  to  facilitate  technology 
transition. 
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TECHNOLOGY  TRANSITION:  GUIDANCE  VERSUS  PRACTICE 


I.  INTRODUCTION 

1.1  BACKGROUND 

Many  definitions  of  technology  transition  are  available  in  the  literature  (Spivey  et 
al,  1997,  Dobbins,  2004,  National  Research  Council,  2004,  A  Guide  for  S&T  Program 
Managers,  2001,  and  The  Manager’s  Guide  to  Technology  Transition  in  an  Evolutionary 
Acquisition  Environment,  2003).  A  definition  from  the  Guide  for  S&T  Program 
Managers,  published  by  the  Deputy  Under  Secretary  of  Defense  (DUSD)  (Science  and 
Technology  Department)  in  2001,  offers  the  following:  “technology  transition  is  the 
process  of  inserting  critical  technology  into  military  systems  to  provide  an  effective 
weapon  and  support  system  -  in  the  quantity  and  quality  needed  by  the  warfighter  to  carry 
out  assigned  missions  -  at  the  best  value  as  measured  by  the  warfighter.”  Technology 
transition  is  just  one  aspect  of  the  overarching  process  in  getting  technologies  to  the 
warfighter.  Other  processes  include  Small  Business  Innovation  Research  (SBIR), 

Defense  Advanced  Research  Projects  Agency  (DARPA),  and  typical  acquisition 
development.  The  transition  of  technologies  from  the  Air  Force  Research  Laboratory 
(AFRL)  to  the  acquisition  community  will  be  the  focus  of  this  research. 

For  purposes  of  clarification  throughout  this  document,  the  tenn  “warfighter”  or 
“user”  reference  those  which  acquire  technologies  to  be  inserted  into  defense  systems. 
When  a  technology  reaches  a  certain  maturity  point  to  be  classified  as  a  6.3  program 
(Table  1),  it  is  capable  of  being  transitioned  to  the  acquisition  community.  The  maturity 
levels  for  technologies  will  be  discussed  in  depth  in  later  chapters.  Technologies  being 
developed  in  AFRL  are  differentiated  by  numerical  designations.  This  study  will 
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concentrate  only  on  6.3  projects.  Typically  technologies  that  are  in  the  6.1  and  6.2 
categories  are  not  mature  enough  to  transition.  Before  a  project  is  given  a  6.3 
designation,  the  project  must  be  able  to  demonstrate  a  certain  capability  and  have  a 
defined  customer.  Research  projects  that  have  a  6.3  designation  normally  examine 
existing  systems  (legacy  systems)  that  the  new  technology  can  be  inserted  into.  Table  1 
illustrates  the  different  classifications  for  projects  within  AFRL. 


Table  1  Technology  Designations 


Community 

Numerical 

designation 

Category 

Science  and 
Technology 

6.1 

Basic  research 

6  2 

Applied  research 

6  3 

Advanced  technology  development 

Acquisition 

6  4 

Demonstration  and  validation 

6.5 

Engineering  and  manufacturing  development 

6  6 

Management  support 

6  7 

Operational  systems  development 

Technology  transition  is  critical  because  it  assists  in  the  delivery  of  advanced 
technological  warfighting  systems  to  the  end  user.  AFRL  is  an  organization  that  plays  a 
keys  role  in  the  development  of  advanced  technologies.  AFRL  transitioning  advanced 
technologies  ensures  the  warfighter  has  superior  technology  to  support  their  missions, 
and  delivers  them  revolutionary  “war-winning”  capabilities  (Deputy  Under  Secretary  of 
Defense,  2001). 

AFRL  demonstrates  through  the  development  of  advanced  technologies,  that  they 
hold  a  position  of  distinct  significance  in  the  development,  demonstration,  and  transition 
of  technologies.  As  evidenced  by  the  following  statement,  experimenting  with  advanced 
innovative  technologies  such  as  remote  sensing,  long-range  precision,  homeland  defense, 
and  infonnation  operations,  illustrates  the  importance  of  AFRL  in  view  of  the  fact  that 
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they  are  intimately  involved  with  developing  technologies  that  give  the  warfighter  critical 
capabilities. 

“We  must  prepare  for  more  such  deployments  by  developing  assets  such 
as  advanced  remote  sensing,  long-range  precision  strike  capabilities,  and 
transformed  maneuver  and  expeditionary  forces.  This  broad  portfolio  of 
military  capabilities  must  also  include  the  ability  to  defend  the  homeland, 
conduct  information  operations,  ensure  access  to  distant  theaters,  and 
protect  critical  U.S.  infrastructure  and  assets  in  outer  space.  Innovation 
within  the  anned  forces  will 

rest  on  experimentation  with  new  approaches  to  warfare,  strengthening 
joint  operations,  exploiting  U.S.  intelligence  advantages  and  taking  full 
advantage  of  science  and  technology.” 

-The  National  Security  Strategy  of  the  United  States,  September  2002 
Technology  transition  is  a  process  which  takes  technologies  from  the  Science  and 
Technology  (S&T)  community  to  a  program  office.  This  transition  of  technology  can  be 
the  reason  for  a  new  start  program,  or  can  be  inserted  into  a  legacy  system,  which  are 
systems  currently  managed  by  a  program  office.  Technology  transition  only  works  if 
AFRL  and  the  acquisition  communities  have  intimate  knowledge  on  the  technology 
(Technology  Transition  for  Affordability;  A  Guide  for  S&T  Program  Managers,  2001). 
The  acquisition  communities  include  both  the  program  offices  and  Major  Commands 
(MAJCOMs).  The  key  stakeholders  in  the  technology  transition  process  are  the  research 
laboratories,  MAJCOMs,  program  offices,  and  the  prime  contractor.  S&T  program 
managers  (PMs)  need  to  collaborate  with  these  organizations  to  facilitate  technology 
transition.  These  communities  may  all  have  different  goals,  time  lines,  and  funding 
levels,  but  they  still  must  find  a  way  to  collaborate  during  different  phases  of  technology 
transition  (National  Research  Council,  2003).  The  collaboration  is  critical  to  achieve 
success  (National  Research  Council,  2003). 
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The  importance  of  technology  transition  stems  from  senior  leaders  placing 
emphasis  on  military  organizations  to  convey  technologies  to  the  warfighter  faster.  If 
these  three  key  stakeholders  (AFRL,  Program  Office,  and  MAJCOM)  do  not  work 
together  the  technology  often  goes  nowhere  (A  Guide  for  S&T  Program  Managers, 
2001).  In  2001,  the  Secretary  of  Defense  stated  the  transition  of  technologies  from  the 
S&T  community  to  the  acquisition  workforce  needed  to  be  improved  by  these 
communities  working  more  closely  together. 

Technology  transition  should  not  be  confused  with  technology  transfer. 
Technology  transfer  is  a  partnership  which  facilitates  the  sharing  of  advanced 
technologies  between  the  government  and  commercial  industry  (Dobbins,  2004). 
Technology  transfer  will  not  be  addressed  in  this  research. 

1.2  THE  PROBLEM 

Efforts  to  bridge  the  gap  between  the  research  laboratories  and  the  warfighter, 
referred  to  by  many  as  the  valley  of  death,  remains  an  issue  for  AFRL  and  acquisition 
communities.  The  valley  of  death  refers  to  technologies  that  are  not  successfully 
transitioned.  The  failure  to  transition  can  be  caused  by  a  lack  of  necessary  buy  in  from 
organizations  needed  to  transition  AFRL  projects.  Other  issues  such  as  the  lack  of 
funding  necessary  to  transition  technologies  can  cause  them  to  get  stuck  in  the  valley  of 
death.  Some  organizations  attribute  technologies  getting  stuck  in  the  valley  of  death  to 
the  different  perceptions  that  exist  between  AFRL  and  acquisition  communities,  while 
others  see  it  as  not  enough  cooperation  among  the  different  communities,  i.e.  end  user, 
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MAJCOMs,  defense  contractor,  and  program  offices  (Defense  Systems  Acquisition 
Management  Course,  2004  and  The  Manager’s  Guide  to  Technology  Transition  in  an 
Evolutionary  Acquisition  Environment,  2003). 

The  technology  transition  process  is  comprised  of  a  distinct  series  of  actions  that 
each  requires  a  different  method  to  successfully  transition  technology.  Figure  1 
illustrates  the  key  issues  associated  with  the  technology  transition  process.  Perceptions, 
funding,  and  different  cultures  pose  many  problems  and  may  cause  for  a  technology  to 
get  stuck  in  the  valley  of  death.  There  are  also  different  actions  that  must  take  place  to 
transition  technologies  within  the  departments:  Basic  Research,  Applied  Research,  and 
Advanced  Technology  Development.  The  critical  set  of  actions,  which  will  be  the  focus 
of  this  study,  will  concentrate  on  technologies  in  the  Advanced  Technology  Development 
stage.  This  is  also  the  stage  of  development  where  the  project  goes  from  a  6.3 
designation  to  a  6.4.  These  actions  include:  stakeholder  collaboration,  Applied 
Technology  Demonstration  (ATD)  review  process,  financial  or  budget  events,  and 
resource  management. 
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Perceptions"  of  the  S&T  Community 
S&T's  job  is  complete  at  the  tech 
development  stage 

Implementation  of  the  technology  is  the 
customer  s  (problem)  responsibility 
The  role  of  S&T  is  "tech  push” —  If  it’s 
good  technology  —  they  will  come! 
Development  cycle  for  S&T  is  too  long  for 
most  Acquisition  and  Warfighter 
customers 

Focus  only  on  the  technology  and  not  on 
the  business  rationale  for  implementation 


Key  Impediments 

•  Budget:  Lack  of  Transition 
Funds 

•  Transition  Process  Lacks 
Definition  &  Visibility 

•  Culture:  Difference  Gcals  & 
Timelines  between  S&T  and 
Acquisition  Managers 

•  Lack  of  Incentives 


7/I/t/W/g/t/f/Jf/A 


'S&T 


Basic 

Research 


Applied 

Research 


Adv  Tech 
Dev 


Adv  Comp  Dev 
&  Prototypes 


Systems 
Dev  &  Demo 


Op  System 
Dev 


Tech  Base 
Managed  by  Labs ■ 


Managed  by 

System  Program  Offices 


Technology  Transition  “Seam" 


(Defense  Systems  Acquisition  Management  Course,  2004) 

Figure  1  Valley  of  Death 

As  Figure  1  illustrates,  funding  is  a  major  impediment  in  bridging  the 
valley  of  death.  Because  of  events  such  as  September  1 1  and  the  current  war  in  Iraq,  the 
Department  of  Defense  (DOD)  budget  is  constrained  (Deputy  Under  Secretary  of 
Defense,  2001).  The  constancy  of  funding  for  technology  available  for  the  demonstration 
stage  all  the  way  through  to  the  full  maturity  level  is  seldom  available  (National  Research 
Council,  2004).  Funding  needs  to  be  available  to  transition  technologies  from  the 
Science  &  Technology  community  to  the  acquisition  workforce. 

To  keep  pace  with  commercially  developed  advanced  technologies,  DOD 
products  need  to  be  rapidly  transitioned  to  the  operational  community  (Deputy  Under 
Secretary  of  Defense,  2001).  AFRL  must  keep  pace  with  the  commercial  sector  to  ensure 
cutting  edge  technologies  are  transitioned  to  the  warfighter  faster  than  typical  acquisition 
development  times.  If  AFRU  does  not  keep  pace  with  the  commercial  sector  then  they 
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are  not  delivering  advanced  technologies.  This  is  critical  because  the  average  acquisition 
development  time  ranges  from  6.5  to  1 1  years,  which  by  that  time  the  technology  is  often 
outdated  (McNutt,  1998).  A  delay  of  technology  transition  may  result  in  warfighters 
possessing  outdated  systems  causing  an  inability  to  respond  or  defend  against  increasing 
threats  of  the  enemy. 

By  the  time  it  takes  a  technology  to  be  developed  and  fielded  to  a  military  unit, 
i.e.  end  state,  the  technology  is  no  longer  state  of  the  art  (McNutt,  1998).  Quite  often 
these  technologies  being  delivered  to  the  warfighter  are  obsolete  (McNutt,  1998).  The 
rapid  transition  of  critical  technologies  to  the  warfighter  is  also  an  effort  to  reduce  long 
development  times  (SECDEF,  2001).  If  the  warfighter  cannot  leverage  up-to-date 
technologies,  then  obsolete  technologies  may  hinder  operational  effectiveness.  AFRL 
transitioning  advanced  technologies  is  just  one  effort  in  ensuring  obsolete  technologies 
are  not  delivered  to  the  warfighter. 

1.3  THE  PURPOSE 

The  purpose  of  this  study  is  to  discover  to  what  extent  AFRL  is  implementing 
known  technology  transition  best  practices.  This  research  will  look  at  the  current 
guidance  on  technology  transition  and  detennine  if  it  is  presently  being  used  by  program 
managers  within  AFRL.  This  study  will  consider  the  technology  transition  process  or 
processes  expressly  exercised  within  AFRL.  In  most  companies  today,  no  one  is  in 
charge  of  the  processes,  and  hardly  anyone  is  even  aware  of  them  (Hammer,  2003).  An 
understanding  of  this  process  from  the  S&T  program  manager’s  perspective  should  give 
insight  into  the  implementation  of  technology  transition  guidance,  transition  roadblocks, 
collaboration  methods,  and  views  on  technology  transition  guidance. 
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The  best  practices  being  evaluated  come  from  Technology  Transition  for 
Affordability;  A  Guide  for  S&T  Program  Managers,  DOD  5000.1,  DOD  5000.2-R,  and 
DOD  sponsored  publications  on  technology  transition.  The  phenomenon  of  technology 
transition  from  AFRL  to  the  Air  Force  warfighter,  as  it  compares  to  best  practice 
guidance,  will  be  the  focus  of  this  research.  This  research  will  try  to  determine  how 
effective  the  technology  transition  guidance  is.  This  will  be  accomplished  through 
examination  of  the  individuals,  processes,  organizations,  and  events  within  AFRL. 

There  has  not  been  much  research  to  date  on  the  topic  of  technology  transition 
within  AFRL.  Although  there  is  considerable  literature  on  various  aspects  of  technology 
transition,  there  is  a  paucity  of  published  research  on  this  topic  (Spivey  et  al,  1997).  This 
study  will  focus  on  the  transition  of  technology  for  new  start  programs  as  well  as 
upgrades  to  legacy  systems.  Nine  best  practices  for  technology  transition  commonly 
used  by  the  DOD  and  industry  will  be  addressed.  These  best  practices  will  be  used  to 
evaluate  the  extent  AFRL  is  implementing  known  technology  transition  best  practices 
from  industry  and  DOD  standards,  in  order  to  accelerate  the  transition  of  laboratory 
technologies  to  the  warfighter. 
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II.  LITERATURE  REVIEW 


2.1  GUIDELINES  FOR  TECHNOLOGY  TRANSITION 

The  best  practices  that  serve  as  the  focus  for  this  study  come  from  the  following 
documents:  Technology  Transition  for  Affordability:  A  Guide  for  S&T  Program 
Managers;  DOD  5000.1;  DOD  5000.2;  Managers  Guide  to  Technology  Transition  in  an 
Evolutionary  Acquisition  Environment;  and  Accelerating  Technology  Transition; 
Bridging  the  Valley  of  Death  for  Materials  and  Processes  in  Defense  Systems. 

Combined,  these  documents  lay  out  effective  steps  an  S&T  manager  can  take  to  facilitate 
technology  transition. 

The  Technology  Transition  for  Affordability ;  A  Guide  for  S&T  Program 
Managers  is  written  to  serve  as  a  guide  to  transition  superior  technologies  while 
considering  cost,  budget,  and  funding  strategies.  The  rapid  transition  for  affordability  is 
essential  to  stay  ahead  of  military  adversaries  who  might  have  the  capabilities  of 
purchasing  advanced  technologies  such  as  weapons  of  mass  destruction.  The  focus  of 
this  document  is  the  affordability  aspect  of  technology  transition.  The  document  also 
outlines  key  elements,  guidelines,  and  DOD  initiatives  that  are  currently  underway  to 
help  assist  technology  transition. 

Affordability  is  referred  to  as  the  delivery  of  a  quality  product  to  the  warfighter  at 
best  value  (Technology  Transition  for  Affordability;  A  Guide  for  S&T  Program 
Managers,  2001).  To  attain  the  stamp  of  best  value,  a  technology  must  increase 
performance,  and  lower  the  cost  of  development,  production,  acquisition,  and  life-cycle 
operations  (Technology  Transition  for  Affordability;  A  Guide  for  S&T  Program 
Managers,  2001).  Technology  transition  relies  on  budgets  and  transition  funding.  If 
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there  were  unlimited  budgets,  then  there  would  not  be  a  budget  or  funding  issue.  The 
purpose  for  this  document  is  to  get  S&T  program  managers  to  consider  project  costs, 
specifically  transition  funding.  However,  this  is  not  possible  unless  the  S&T  PM  teams 
with  the  acquisition  customer  early  on  in  the  project.  It  is  essential  that  the  acquisition 
community  plans  for  the  transition  early  on  and  sets  aside  transition  dollars.  It  is  also 
ideal  that  the  S&T  and  acquisition  communities  establish  a  partnership  during  Milestone 

A,  which  is  the  pre-systems  acquisition  phase  (DOD  5000.1). 

The  DOD  5000  series,  which  include  5000.1,  5000.2,  and  5000. 2-R,  attempt  to 
provide  guidelines  for  the  insertion  of  S&T  products.  The  documents  are  very  broad 
when  discussing  technology  transition.  The  5000  series  documents  place  emphasis  on 
affordability,  as  well  as  mechanisms  that  ensure  mature  technologies  are  transitioned. 
Technology  Readiness  Levels  (TRL),  as  well  as  demonstrating  technologies  in  a  relevant 
environment,  is  the  main  mechanism  used  to  detennine  technology  maturity.  Technology 
developed  in  the  S&T  community  should  be  demonstrated  in  a  relevant  environment  or  in 
an  operational  environment  to  be  considered  mature  enough  to  transition  (Section  3.7 .2.2 
DOD  5000. 2-R). 

The  DOD  5000  series  documents  also  discuss  the  importance  of  rapidly 
transitioning  S&T  products  to  acquisition  programs.  Figure  2  illustrates  key  points  from 
the  5000  series  documentation.  As  stated  earlier,  it  is  ideal  for  the  S&T  and  acquisition 
communities  to  focus  on  Milestone  A,  however,  S&T  products  can  enter  at  Milestones  A, 

B,  or  C  depending  on  the  program  state  and  transition  windows  (DOD  5000.2-R). 
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THE  5000  MODEL 


DoDD  50011. 1 

•  Rapid  Transition  From 
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•  Emphasis  on  Affordability 
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•  Focus  on  SAT  Solutions  in 
Pre  -Acquisition 

•  Use  Mechanisms  with  User 
A  Acquisition  C ustomer  to 
Ensure  Transition 

Do  I)  5000 J-R 

•  Establish  Technology 
Readiness  Lev  els  for 
Critical  Technologies 


Technology  Opportunities 
&  User  Needs 


•  Process  entry  at  Milestones 
.4,  B.  or  C  (or  within  phases) 

•  Program  outyear  funding 
when  it  makes  sense,  hut  no 
later  Ilian  Milestone  B 


A  A  A 
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Concept  & 

Technology 

Development 

System 
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4  Demonstration 

Production  & 
Deployment 

Support 

Pre-Systems 

Acquisition 

Systems  Acquisition 
lEngfneerliyi  Development. 
Demonstration.  LRIP  &  Production i 

Sustalnments 

Maintenance 

(Technology  Transition  for  Affordability;  A  Guide  for  S&T  Program  Managers,  2001) 

Figure  2  DOD  5000  Series  Snapshot 

The  Manager’s  Guide  to  Technology  Transition  in  an  Evolutionary  Acquisition 
Environment  is  another  important  document  from  which  technology  transition  best 
practices  were  derived.  This  document  is  not  intended  to  define  a  specific  technology 
transition  process.  It  is  simply  to  be  used  by  S&T  program  managers  as  a  reference 
guide.  The  document  also  suggests  that  the  program  manager  refer  to  their  immediate 
sources  of  authority  when  using  technology  transition  processes  (Manager’s  Guide  to 
Technology  Transition  in  an  Evolutionary  Acquisition  Environment,  2003).  This  guide  is 
intended  to  illustrate  how  different  communities  can  collaborate  to  promote  technology 
transition.  The  document  also  addresses  processes,  programs,  and  challenges  associated 
with  technology  transition. 

According  to  the  Manager’s  Guide  to  Technology  Transition  in  an  Evolutionary 
Acquisition  Environment  document,  there  are  three  main  communities  identified  that 
should  be  involved  in  facilitating  technology  transition.  These  three  players  are  the 
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requirements  generation,  S&T,  and  acquisition  communities.  The  requirement 
community  is  responsible  for  developing  requirement  documents  that  define  full 
functionality  of  a  system  that  meets  warfighters  needs  at  the  beginning  of  a  program.  In 
other  words,  they  determine  exactly  what  the  user  needs  are.  Another  approach  is 
defining  capabilities  based  on  the  maturity  of  the  technology.  This  is  a  new  approach  for 
the  requirements  community,  which  needs  close  interaction  between  the  program  offices 
and  S&T  community.  The  requirements  generation  community  is  also  responsible  for  the 
analysis  of  mission  capabilities.  This  analysis  gives  other  communities,  such  as  the  S&T 
community,  more  involvement  in  the  requirements  generation  process.  This  is  key 
because  it  identifies  opportunities  to  exploit  advanced  technologies  that  could  meet 
warfighter  needs  (Manager’s  Guide  to  Technology  Transition  in  an  Evolutionary 
Acquisition  Environment,  2003). 

The  acquisition  community  also  plays  a  major  role  in  the  implementation  of 
technology  transition.  Evolutionary  acquisition  provides  the  warfighter  capabilities 
through  a  spiral  approach  (i.e.  increments).  This  is  the  preferred  method  to  facilitate  the 
rapid  transition  of  mature  technologies  (Manager’s  Guide  to  Technology  Transition  in  an 
Evolutionary  Acquisition  Environment,  2003).  The  S&T  PM  must  attempt  to  get  the 
necessary  buy  in  from  the  program  offices  to  ensure  key  stakeholders,  such  as  the 
acquisition  community,  are  aware  of  what  is  going  on. 

The  third  major  player  in  technology  transition  is  the  financial  management 
community.  Fiscal  limits  are  placed  on  budgets  every  year  though  the  Defense  Planning 
Guidance  (DPG),  which  are  promulgated  by  the  Secretary  of  Defense.  This  forces 
program  offices  to  develop  the  Program  Objectives  Memorandum  (POM)  within  a 
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specific  fiscal  year.  Program  offices  must  budget  for  a  set  amount  of  money,  which  is 
tailored  to  the  development  of  a  specific  program.  It  is  important  to  understand  this 
process  and  how  the  budget  process  affects  technology  transition.  This  process  forces 
organizations  such  as  the  S&T  community  to  delegate  different  funds  (colors  of  money) 
to  the  development  of  technologies  (Manager’s  Guide  to  Technology  Transition  in  an 
Evolutionary  Acquisition  Environment,  2003).  The  colors  of  money  represent  dollars 
that  are  specifically  set  aside  for  6.1,  6.2,  and  6.3  programs.  This  does  not  allow  the 
managers  the  ability  to  shift  funds  around  as  they  see  fit.  Program  managers  of  6.3 
projects  can  only  use  money  for  6.3  efforts.  This  means  mature  technologies  have  a 
limited  amount  of  funds  to  get  it  to  the  transition  point.  The  transition  point  is  when  an 
acquisition  community  or  program  office  is  willing  to  take  ownership  of  the  technology. 

Other  key  players  in  technology  transition  that  are  discussed  in  The  Manager’s 
Guide  to  Technology  Transition  in  an  Evolutionary  Acquisition  Environment  guide  are 
the  Research  and  Development  (R&D)  and  S&T  communities.  The  S&T  community 
should  focus  on  transitioning  advanced  technologies  to  affordable  products  and  during 
the  process  partner  with  the  acquisition  community  to  fully  address  the  warfighters  needs 
(Manager’s  Guide  to  Technology  Transition  in  an  Evolutionary  Acquisition 
Environment,  2003).  This  process  is  typically  accomplished  through  Applied 
Technology  Demonstrations  (ATD)  and  Applied  Concept  Technology  Demonstrations 
(ACTD).  The  R&D  community  evaluates  technologies  and  is  responsible  for  getting 
them  to  the  field  by  working  with  the  acquisition  communities  (Manager’s  Guide  to 
Technology  Transition  in  an  Evolutionary  Acquisition  Environment,  2003).  This 
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community  also  focuses  on  interoperability  and  interface  issues  of  systems,  so  the 
warfighter  receives  a  user  friendly  product  that  meets  their  needs. 

The  last  document  that  gave  tremendous  insight  to  technology  transition  was  the 
publication  Accelerating  Technology  Transition;  Bridging  the  Valley  of  Death  for 
Materials  and  Processes  in  Defense  Systems.  In  2003,  the  DOD  requested  that  the 
National  Research  Council  (NRC)  sponsor  a  workshop  that  focused  on  a  variety  of 
aspects  that  influence  technology  transition.  The  three  main  focus  areas  were:  cultures 
that  fostered  an  environment  to  transition  technology;  methodologies  and  approaches  to 
technology  transition;  and  tools  that  enable  technology  transition  (National  Research 
Council,  2004). 

One  key  aspect  to  creating  the  right  culture  is  to  foster  communication  through 
the  development  of  prototypes.  This  is  done  by  putting  the  prototypes  in  the  hands  of  the 
customer  as  early  as  possible  (National  Research  Council,  2004).  The  customer  can  get  a 
better  understanding  of  the  capability  of  the  technology  by  allowing  them  to  interact  with 
the  technology.  This  essentially  fosters  communication  among  all  the  stakeholders, 
allowing  them  to  have  a  common  understanding  of  the  technology  potential. 

Breaking  down  cultural  barriers  was  another  essential  step  to  creating  the  right 
culture.  There  are  hierarchical  organizations  that  have  cultural  traits  which  hinder 
technology  transition.  An  example  would  be  a  program  office  that  might  be  in  need  of  a 
specific  technology  in  which  they  automatically  pursue  a  solution  through  a  defense 
contractor,  bypassing  possible  solutions  that  might  exist  in  the  S&T  community. 

The  second  significant  aspect  to  technology  transition  that  the  NRC  found  was  the 
methodologies  and  approaches  used  to  transition  a  product.  One  methodology  is  to 
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include  a  viral  process  which  incorporates  knowledge  into  the  development  process 
(National  Research  Council,  2004).  A  viral  process  is  established  when  S&T  PMs 
become  knowledgeable  about  available  tools  that  can  help  accelerate  technology 
transition.  These  tools  might  include  software  capabilities  that  allow  for  advanced 
modeling  and  prototyping.  The  PM  must  know  what  resources  are  available  to  facilitate 
the  transition  of  a  technology. 

Another  methodology,  or  approach,  found  by  the  NRC  was  to  foster  technology 
transition  through  the  creation  of  a  successful  team.  For  a  PM  to  successfully  transition 
technology,  they  must  have  the  right  technologies,  right  team  skills,  and  the  right 
financial  support  (National  Research  Council,  2004).  For  the  teams  to  be  successful  the 
customer  must  be  involved  at  the  beginning.  Also,  if  these  teams  do  not  have  support 
from  the  highest  level,  then  they  will  be  unsuccessful  (National  Research  Council,  2004). 
The  formation  of  these  teams  should  foster  an  environment  focused  on  maturing 
technologies  (National  Research  Council,  2004).  A  transition  team  focused  on  the 
maturity  and  reliability  of  the  technology,  which  can  be  accomplished  through  multiple 
experimentations  and  demonstrations  in  challenging  environments,  can  meet  the  demands 
of  acquisition,  testing,  and  evaluation  organizations  (Institute  for  Defense  Analysis, 

2003).  This  approach  is  also  in  accordance  with  the  DOD  5000.2-R. 

Another  area  of  concern  in  this  study  was  the  use  of  enabling  tools  and  databases. 
The  tools  discussed  were  the  Accelerated  Insertion  of  Materials  (AIM)  and  Materials 
Engineering  of  Affordable  New  Systems  (MEANS).  Research  performed  by  the  NRC 
was  focused  on  the  rapid  transition  of  materials,  which  is  why  the  AIM  and  MEANS 
tools  were  discussed.  Although  these  tools  focus  on  material  transition,  it  does  not  mean 
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the  concept  can  not  be  applied  to  the  overall  notion  of  technology  transition.  Databases 
that  can  be  shared  and  accessed  by  all  the  key  stakeholders  allows  for  better 
communication  and  understanding  of  that  specific  technology.  Collaborative  tools  can 
enable  an  environment  which  fosters  an  innovative  culture  that  can  focus  on  the  rapid 
transition  of  advanced  technologies. 

2.2  BEST  PRACTICES 
Communication 

Communication  and  ownership  of  technologies  between  the  S&T  community  and 
user  is  vital  for  technology  transition  to  be  successful  (Spivey  et  al,  1997,  and  Manager’s 
Guide  to  Technology  Transition  in  an  Evolutionary  Acquisition  Environment,  2003). 
Throughout  the  DOD  5000,  and  other  DOD  sponsored  documents,  the  importance  of 
communication  between  the  laboratories  and  the  acquisition  personnel  is  stressed 
repeatedly  (The  Manager’s  Guide  to  Technology  Transition  in  an  Evolutionary 
Acquisition  Environment,  2003,  National  Research  Council,  2004  and  Technology 
Transition  for  Affordability;  A  Guide  for  S&T  Program  Managers,  2001).  While 
communication  is  just  one  of  many  key  aspects  in  transitioning  technology  from  one 
organization  to  another,  technology  can  be  stymied  in  its  transition  if  communication 
between  the  S&T  community  and  acquisition  personnel  is  intermittent  (Spivey  et  al, 
1997).  If  some  players  lack  effective  communication  it  can  lead  to  the  philosophy  that 
the  technology  was  “not  invented  here”  and  question  why  it  should  be  championed 
(Spivey  et  al,  1997). 

Communication  is  an  integral  part  of  technology  transition.  Some  authors  find 
communication  to  be  the  most  important  condition  for  success  (Hoetker,  1997; 
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Lundquist,  1999;  Russo  and  Herrenkohl,  1990).  Communication  was  established  as  the 
first  best  practice.  Developing  a  strategy  and  planning  for  technology  transition  involves 
many  departments  internal  to  AFRL  and  external  which  include  MAJCOMs,  program 
offices,  and  defense  contractors.  There  can  not  be  success  if  the  collaboration  lacks 
effective  communication  (Spivey  et  al,  1997,  Dobbins,  2004,  National  Research  Council, 
2004,  A  Guide  for  S&T  Program  Managers,  2001,  and  The  Manager’s  Guide  to 
Technology  Transition  in  an  Evolutionary  Acquisition  Environment,  2003).  To  make  a 
smooth  transition,  the  S&T  and  acquisition  communities  must  communicate  early  and 
often  (Manager’s  Guide  to  Technology  Transition  in  an  Evolutionary  Acquisition 
Environment,  2003,  National  Research  Council,  2004,  Technology  Transition  for 
Affordability:  A  Guide  for  S&T  Program  Managers,  2001).  The  Managers  Guide  to 
Technology  Transition  in  an  Evolutionary  Acquisition  Environment  also  suggests  ways 
to  keep  in  close  contact  with  the  customer,  as  well  as  ways  to  make  pertinent  information 
about  the  technology  readily  available.  Using  resources  such  as  web  sites,  published 
articles,  conventions,  briefings,  and  shared  databases,  in  conjunction  with  working 
directly  with  the  program  offices,  enables  the  S&T  community  to  effectively 
communicate  with  the  customer.  Well  thought  out  strategies  and  communication  allows 
ah  the  participants  to  have  an  understanding  of  the  project  schedule,  as  well  as  the 
maturity  of  the  technology.  Technologies  can  only  transition  if  the  key  stakeholders 
communicate,  roles  and  responsibilities  are  defined,  and  funding  is  consistent  (Manager’s 
Guide  to  Technology  Transition  in  an  Evolutionary  Acquisition  Environment,  2003). 
Proper  and  effective  communication  should  facilitate  technology  transition  and  get  buy  in 
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from  senior  leaders  and  necessary  stakeholders  such  as  the  MAJCOMs,  Program  Offices, 
and  end  user  (National  Research  Council,  2004). 

It  is  important  to  get  buy  in  from  upper  management.  Top-level  managers  should 
motivate  personnel  at  every  level  to  clearly  identify  objectives  and  identify  weapon 
system  program  office  needs  in  order  to  plan  an  effective  S&T  program  that  focuses  on 
technology  transition  (A  Guide  for  S&T  Program  Managers,  2001).  The  transition  of  a 
technology  may  be  thwarted  if  there  is  no  senior  management  support.  Also,  without 
support  from  upper-management,  PMs  must  navigate  through  organizational  barriers  at 
the  low  and  high  levels  within  their  organization  as  well  as  with  outside  organizations 
such  as  MAJCOMs  and  program  offices.  An  additional  and  essential  factor  is  a 
champion  with  sufficient  authority  to  remove  barriers,  garner  support,  and  ensure  a  new 
technology’s  successful  implementation  and  use  (National  Research  Council,  2004).  If 
the  S&T  PM  does  not  have  buy  in,  then  they  do  not  have  senior  leaders  willing  to 
champion  the  technology.  In  most  cases,  the  champion  is  typically  a  MAJCOM 
representative  or  program  office  willing  to  agree  to  take  over  program  management 
responsibilities,  as  well  as  set  aside  transition  money  for  the  technology  to  be 
transitioned.  If  the  PMs  of  6.3  projects  can  work  closely  with  the  MAJCOMs  and 
program  offices,  a  natural  team  environment  can  develop.  This  outcome  might 
eventually  foster  a  culture  focused  on  rapidly  transitioning  innovative  technologies, 
which  was  a  best  practice  suggested  by  the  National  Research  Council. 

Identifying  the  Customer 

When  a  PM  attempts  to  get  buy  in  for  their  technology,  they  should  have  a 
customer  identified.  Identifying  your  customer  and  knowing  what  organizations  frame 
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your  customer  is  the  second  best  practice  for  technology  transition.  The  customer 
performs  a  vital  role  in  transitioning  technologies.  Typically,  people  involved  in  a 
process  look  inward  toward  their  department  and  upward  to  their  boss,  but  no  one  looks 
outward  directly  toward  the  customer  (Hammer  et  al,  2003).  There  are  different  views  on 
who  the  customer  is.  Some  view  the  customer  as  the  MAJCOM,  the  end  user,  while 
others  view  the  customer  as  the  contractor  or  the  program  office.  The  eventual  customer 
will  always  be  the  warfighter.  The  PM  must  understand  the  role  of  a  customer  as  well  as 
the  importance  of  recognizing  the  customer  early  on. 

Participation 

In  dealing  with  the  customer,  one  realizes  that  a  key  factor  to  successful 
technology  transition  lies  in  the  ability  to  team  with  the  customer.  Participation  is  the 
third  best  practice  that  will  be  addressed.  The  S&T  manager  must  work  in  partnership 
with  the  customer  on  schedules,  performance,  costs,  and  other  key  areas  that  play  a  role 
in  technology  transition  (A  Guide  for  S&T  Program  Managers,  2001).  An  organization 
such  as  AFRL  needs  to  partner  with  the  acquisition  community  as  well  as  the  MAJCOMs 
to  ensure  advanced  technologies  transition  to  the  warfighter.  Partnering  becomes  a  major 
factor  because  one  organization  cannot  be  responsible  for  transitioning  technologies. 
Partnership  should  go  above  and  beyond  everyday  functional  roles,  which  promotes  an 
environment  where  the  stakeholders  are  flexible  and  willing  to  take  risks  without  regard 
to  organizational  barriers  (National  Research  Council,  2004).  Teamwork  and 
collaboration  should  take  precedence  over  individual  accomplishments  and  recognition 
(National  Research  Council,  2004).  To  accomplish  participation  and  collaboration,  the 
PM  must  have  the  right  skills,  abilities,  and  resources.  The  PM  must  have  the  necessary 
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information  technology  (IT)  tools  available  and  be  knowledgeable  in  the  necessary 
disciplines  to  be  able  to  communicate  and  collaborate  with  the  customer. 

Information  Technology  (IT) 

Using  IT  tools  as  an  enabler  to  transition  technology  is  the  fourth  best  practice 
that  facilitates  technology  transition.  One  example  of  IT  tools  that  can  enable  the  S&T 
PM  is  being  implemented  by  the  Director  Defense  Research  and  Engineering  (DDR&E) 
who  is  planning  to  develop  an  information  resource  that  gives  all  internal  defense 
technology  providers,  such  as  AFRL,  access  to  most  of  the  current  DOD  projects 
(Manager’s  Guide  to  Technology  Transition  in  an  Evolutionary  Acquisition 
Environment,  2003).  PMs  within  the  S&T  community  will  be  able  to  access  a  large 
number  of  current  programs  which  could  give  them  insight  into  weapon  systems  their 
technology  might  be  able  to  interface  with.  This  also  allows  the  PM  the  opportunity  to 
make  contact  with  possible  customers,  facilitating  communication.  IT  tools,  such  as 
shareable  databases,  allow  the  PMs  to  disseminate  pertinent  information  about  their 
project.  It  is  important  that  the  key  stakeholders  have  access  to  the  databases  so  there  is 
mutual  understanding  on  the  technology  capabilities.  The  Virtual  Technology  Exposition 
(VTE)  is  another  IT  resource  that  S&T  PMs  have  access  to.  The  VTE  is  a  website  that 
contains  subject  areas  on  defense  technology  and  joint  warfighting  capabilities  (A  Guide 
for  S&T  Program  Managers,  2001).  The  VTE  website  can  be  found  at 
https://vte.dtic.mil/. 

Hammer  (2003)  sums  up  the  importance  of  IT  well  when  he  says,  “It  is  the 
disruptive  power  of  technology,  its  ability  to  break  the  rules  how  we  conduct  our  work, 
which  makes  it  critical  to  companies  looking  for  the  competitive  advantage.”  IT  tools 
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can  give  the  S&T  PM  the  advantage  of  collaborating  and  communicating  with  their 
customer  so  technology  can  be  rapidly  transitioned.  These  tools  can  help  cultivate  an 
environment  where  knowledge  is  shared,  which  leads  to  the  fifth  best  practice, 
knowledge  management. 

Knowledge  Management 

Knowledge  management  is  important  for  PMs  to  have  the  ability  to  relate  and 
understand  the  technical  side  and  management  side  of  program  management.  The 
National  Research  Council  discussed  the  importance  for  teams  to  have  the  right 
technologies,  skills,  and  financial  support  in  order  to  implement  technology  transition. 
S&T  PMs  need  to  be  intimately  familiar  with  engineering  and  science  to  understand  the 
technology  capability  and  potential,  as  well  as  management  practices.  The  essence  of 
knowledge  management  is  the  ability  to  share  and  access  information  to  gain  knowledge. 
Being  able  to  share  information  allows  PMs  to  communicate  easier,  think  outside  the 
box,  retrieve  data  more  quickly,  and  avoid  prior  mistakes  (Manager’s  Guide  to 
Technology  Transition  in  an  Evolutionary  Acquisition  Environment,  2003).  To  stay  up 
to  date  with  the  latest  technologies,  the  S&T  PM  must  attend  technical  conferences, 
symposia,  and  academic  meetings  (Manager’s  Guide  to  Technology  Transition  in  an 
Evolutionary  Acquisition  Environment,  2003).  They  must  also  keep  the  acquisition 
community  involved  because  the  program  offices  are  constantly  being  bombarded  with 
information  on  possible  new  technologies  from  defense  contractors.  The  Air  Force 
Knowledge  Now  (AFKN)  website  is  a  good  example  of  a  collaborative  tool  which 
enables  different  communities  to  share  information.  However,  they  also  need  to 
understand  the  financial  aspects  such  as  budget,  POM  cycles,  and  affordability  metrics. 
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These  processes  are  typical  acquisition  functions,  which  can  be  looked  at  as  important 
metrics  that  S&T  PMs  should  be  leveraging  to  facilitate  technology  transition. 
Affordability  Metrics 

Affordability  metrics  and  methodologies  is  the  sixth  best  practice  identified  from 
the  literature.  The  S&T  manager  should  establish  quantitative  metrics  in  order  to  track 
the  progress,  exit  criteria,  and  maturity  of  a  technology  (A  Guide  for  S&T  Program 
Managers,  2001).  It  is  ideal  if  all  the  stakeholders  agree  to  the  metrics  the  S&T  PM  will 
use.  Affordability  metrics  concentrate  on  operating  and  support  costs,  life-cycle  costs, 
and  lead-time  reductions  (A  Guide  for  S&T  Program  Managers,  2001). 

An  affordability  metric  that  can  de  directly  applied  to  technology  transition  is  the 
Integrated  Process  and  Product  Development  (IPPD)  method.  The  IPPD  process  includes 
the  creation  of  Integrated  Product  Teams  (IPTs),  which  are  multidisciplinary  teams  that 
focus  on  meeting  the  customer’s  needs  during  the  development  phase  as  well  as  ensuring 
the  technology  gets  transitioned.  It  is  important  for  the  S&T  PM  to  establish  IPTs  and  an 
IPT  charter  (AFI-61-101).  The  charter  shall  identify  roles  and  responsibilities  for  all  the 
team  members.  This  should  include  the  customer  as  well.  Ensuring  the  right  members 
are  included  in  these  IPTs  is  critical  for  success  of  the  technology.  The  IPPD  process 
also  suggests  that  affordability  metrics  be  used  when  determining  system  perfonnance. 
The  development  of  a  transition  plan  is  another  important  aspect  of  the  IPPD  process. 
Finally,  when  using  the  IPPD  method  the  S&T  PM  must  set  up  a  senior  leader  review 
process. 

These  metrics  and  methodologies,  such  as  experience  with  developing  business 
models  and  Return  on  Investment  (ROI)  strategies,  can  be  somewhat  complex  for  PMs 
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that  have  limited  experience  in  acquisition  processes  and  business  processes.  This  is  why 
it  is  essential  that  the  teams  involved  with  transitioning  technology  have  all  the  necessary 
knowledge,  skills,  and  resources  to  utilize  these  tools. 

Ensuring  Technology  Maturity 

While  PMs  should  understand  acquisition  processes,  they  still  need  to  focus  on 
the  technical  capabilities  and  perfonnance  of  their  6.3  program.  To  do  this  they  must 
constantly  evaluate  and  gauge  the  maturity  of  the  technology.  Ensuring  technology 
maturity  is  the  seventh  best  practice  for  facilitating  technology  transition.  The  National 
Research  Council  and  Government  Accounting  Office  (GAO)  both  found  that  the 
marketing  of  technology  maturity  can  make  or  break  the  transition  of  the  technology. 
Quite  often  early  concepts  are  marketed  as  being  currently  available,  forcing  the 
customer  not  to  purchase  the  existing  product,  but  wait  for  the  new  technology  to  be 
implemented  (National  Research  Council,  2004).  When  this  happens,  an  immature 
concept  is  rushed  to  the  demonstration  phase  where  quite  often  it  fails  to  demonstrate  the 
marketed  capabilities  (GAO,  2002).  When  these  concepts  are  rushed  to  the 
demonstration  phase  they  are  often  left  in  the  valley  of  death.  Technologies  should  be 
demonstrated  at  high  levels  of  maturity  before  new  technologies  are  integrated  (A  Guide 
for  S&T  Program  Managers,  2001). 

One  way  to  ensure  the  maturity  of  technology  is  through  the  use  of  Technology 
Readiness  Levels  (TRLs).  TRLs  serve  as  a  gauge  for  PMs  when  they  need  to  make  a 
determination  on  the  maturity  of  their  technology.  A  technology  that  has  been  designated 
as  a  TRL  4  is  a  lot  less  mature  than  a  technology  at  TRL  6.  All  the  stakeholders  need  to 
understand  the  difference  in  TRLs  because  it  allows  them  to  understand  the  risk  involved 
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with  the  technology.  A  technology  at  TRL  4  will  have  much  more  risk  associated  with  it 
than  a  product  at  TRL  6  or  7.  The  AFI  61-101  also  places  an  emphasis  on  the  use  of 
TRLs,  which  should  be  agreed  on  by  all  the  stakeholders  involved.  The  DOD  5000  series 
documents  suggest  that  technology  must  be  tested  in  an  operational  environment  to  be 
considered  a  mature  technology.  Figure  3  illustrates  the  different  classification  of  TRLs. 
TRL  classifications  also  need  to  be  included  in  the  Technology  Transition  Plan  (TTP). 


Measuring  Technology  Maturity 

Technology  Readiness  Levels 


System  Test.  Launch 
&  Operations 


System/Subsystem 

Development 


Technology 

Demonstration 


TRL  9 
TRL  8 
TRL  7 
TRL  6 


Actual  system  “flight  proven”  through  successful 
mission  operations 

Actual  system  completed  and  “flight  qualified”  through 
test  and  demonstration 

System  prototype  demonstration  in  a  operational 
environment 

System/subsystem  model  or  prototype  demonstration 
in  a  relevant  environment 


Technology 

Development 


Research  to  Prove 
Feasibility 


Basic  Technology 
Research 


Component  and/or  breadboard  validation  in  relevant 
environment 

Component  and/or  breadboard  validation  in  laboratory 
environment 

Analytical  and  experimental  critical  function  and/or 
characteristic  proof-of-concept 

Technology  concept  and/or  application  formulated 


Basic  principles  observed  and  reported 

As  Defined  in  5000.2-R 


(Defense  Systems  Acquisition  Management  Course,  2004) 

Figure  3  TRL  Maturity  Levels 

The  importance  of  having  a  transition  plan  is  that  it  allows  the  S&T  PM  to  get  all 
the  key  stakeholders  to  agree  on  transition  issues  such  as  risk  associated  with  the 
technology  (AFI  61-101).  The  transition  plan  is  only  effective  if  the  key  stakeholders 
communicate  and  collaborate  on  project  issues.  A  transition  strategy  should  allow  the 
S&T  and  acquisition  communities  to  appropriately  plan  for  the  transition  of  a  technology. 
The  best  evidence  of  a  transition  commitment  is  the  inclusion  of  funds  in  either  the  S&T 
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budget  or  acquisition  budget  to  bridge  the  gap  from  S&T  phase  to  the  acquisition  phase 
(A  Guide  for  S&T  Program  Managers,  2001).  Planning  early  on,  especially  with  both 
communities  weighing  the  funding  aspects,  should  help  bridge  the  valley  of  death. 
Technology  Transition  Team 

The  development  of  a  successful  team  provides  a  crucial  foundation  for 
successful  technology  transition  (Dobbins,  2004).  The  implementation  of  a  transition 
team  is  the  eighth  best  practice.  In  an  article  from  Defense  AT&L  (2004),  Dobbins 
explains  the  importance  of  having  a  transition  integrated  product  team  (TIPT).  The 
transition  team  works  to  maintain  a  smooth  transition  of  the  technology  as  well  as 
oversight  responsibility  for  the  transition.  The  TIPT  has  the  capability  of  bringing  the 
MAJCOMs,  program  offices,  laboratories,  and  end  user  together,  ultimately  facilitating 
the  transition  planning  and  other  key  processes  associated  with  technology  transition. 
Having  the  capabilities  of  an  overarching  integrated  product  team  (OIPT)  can  also  help 
provide  for  a  seamless  transition  (Dobbins,  2004).  The  OIPT  is  responsible  for 
completing  the  remainder  of  the  transitioning  activities  such  as  cost,  schedule,  and 
performance.  The  concept  of  people  having  the  right  technologies,  the  right  team  skills, 
and  the  right  financial  support  is  not  new;  additionally,  all  successful  transitions  need  to 
have  the  customers  as  part  of  the  team  from  the  beginning  in  order  to  ensure  the 
military’s  high  performance  requirements  are  being  met  (National  Research  Council, 
2004).  The  S&T  PM  must  have  the  necessary  personnel  that  are  needed  to  transition  the 
technology.  The  program  manager  must  be  able  to  fully  understand  the  technical  and 
financial  aspects  of  their  program.  Program  management  responsibilities  are  quite  often 
new  concepts  that  the  S&T  PM  must  understand.  These  responsibilities  can  be  seen  as 
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additional  duties  they  are  not  used  to  dealing  with,  which  leads  to  the  last  best  practice  of 
using  incentives. 

Incentives 

Incentives  play  an  important  role  in  industry  when  it  comes  to  transitioning  a 
product  to  the  marketplace.  Quite  often  firms  will  offer  cash  bonuses  or  promotions  if 
PMs  can  market  and  transition  their  product  to  commercial  vendors.  However,  incentives 
like  these  do  not  exist  within  AFRL  (Defense  Systems  Acquisition  Management  Course, 
2004).  Other  than  personal  satisfaction,  PMs  do  not  have  any  incentives  to  transition 
their  products. 

Unfortunately,  there  are  disincentives  at  the  senior  leader  level  that  hinders  the 
insertion  of  new  technologies.  At  times,  program  offices  will  have  the  opportunity  to 
adopt  a  new  technology  that  reduces  total  life-cycle  costs,  but  in  the  end  may  turn  down 
the  technology  for  the  reason  that  it  will  lead  to  budget  cuts  for  their  organization 
(Manager’s  Guide  to  Technology  Transition  in  an  Evolutionary  Acquisition 
Environment,  2003).  This  can  be  a  disincentive  for  S&T  PMs  who  are  trying  to 
transition  their  technology. 

2.3  ROADBLOCKS  TO  TECHNOLOGY  TRANSITION 

Technology  transition,  as  with  many  other  processes,  comes  with  barriers  and 
roadblocks  which  hinder  the  process.  In  2003,  the  Office  of  the  Under  Secretary  of 
Defense  (Acquisition,  Technology,  and  Logistics)  published  data  that  described 
challenges  to  technology  transition.  Interoperability  is  listed  as  a  major  concern  when 
transitioning  technologies  from  the  laboratory.  The  necessity  for  interoperability  is  why 
communication  stands  out  as  one  of  the  most  important  aspects  in  technology  transition. 
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If  the  laboratories,  program  offices,  users,  and  contractors  do  not  communicate  early  and 
often,  the  possibility  that  the  technology  will  not  be  able  to  transition  to  the  acquisition 
community  becomes  a  reality.  One  of  the  DOD  5000  series  objectives  is  that  the  PM 
must  focus  on  interoperability  and  supportability.  If  the  technologies  coming  out  of  the 
laboratories  do  not  have  some  type  of  open  system  architecture,  which  allows  for  systems 
already  in  the  battlefield  to  interface  with  the  new  technology,  then  the  technology  might 
again  fall  prey  to  the  valley  of  death  (The  Manager’s  Guide  to  Technology  Transition  in 
an  Evolutionary  Acquisition  Environment,  2003).  Ensuring  for  interoperability  is  vital 
for  a  smooth  transition. 

The  test  environment  was  considered  to  be  another  transition  roadblock  by  the 
OUSD  (AT&L).  To  further  assist  in  a  smooth  transition,  the  test  and  evaluation  (T&E) 
process  must  be  examined.  Sometimes  technologies  are  marketed  as  being  mature  and 
stable  in  a  relevant  environment.  When  an  immature  technology  is  tested  in  a  relevant  or 
operational  environment,  it  quite  often  fails  the  test  phase.  Therefore,  the  technology  is 
left  to  linger  in  the  valley  of  death.  The  T&E  community  provides  an  independent 
assessment  of  the  technology  which  limits  the  chances  of  transitioning  immature 
technologies  to  the  field.  They  ensure  the  technology  works  as  it  is  intended  to  and  can 
stand  up  in  an  operational  environment.  Most  S&T  technologies  that  are  at  a  TRL  4-6 
are  validated  in  a  lab  or  simulated  operational  environment  (Manager’s  Guide  to 
Technology  Transition  in  an  Evolutionary  Acquisition  Environment,  2003). 

Consequently,  funding  might  possibly  be  the  single  most  important  roadblock  to 
technology  transition  (National  Research  Council,  2004,  A  Guide  for  S&T  Program 
Managers,  2001,  and  The  Manager’s  Guide  to  Technology  Transition  in  an  Evolutionary 
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Acquisition  Environment,  2003).  If  the  S&T  community,  MAJCOMs,  or  program  offices 
have  not  set  aside  money  for  transition,  then  the  technology  will  be  very  difficult  to 
transition  from  the  laboratory  to  the  program  office.  Proper  allocation  of  funds  requires 
early  communication  and  buy  in  among  the  stakeholders.  Communication  will  allow 
organizations  an  inroad  of  understanding  to  future  funding  requirements.  If  S&T  PMs 
align  their  projects  with  specific  future  warfighting  needs,  quite  often  they  will  obtain  the 
highest  priority  for  funding  (Manager’s  Guide  to  Technology  Transition  in  an 
Evolutionary  Acquisition  Environment,  2003).  PMs  must  budget  for  three  to  eight  years 
down  the  road. 

Among  the  challenges  to  technology  transition,  cultural  differences  must  be 
included.  The  laboratory,  acquisition,  and  operational  communities  do  not  always  have 
the  same  goals  and  quite  often  have  a  difference  of  opinion  in  operational  tactics,  which 
is  why  communication  and  participation  are  so  important  to  ensure  mutual  understanding 
(National  Research  Council,  2004).  The  senior  leaders  from  these  communities  need  to 
effectively  manage  the  different  cultures  to  ensure  the  technology  is  transitioned  from  the 
development  stage  to  implementation  (National  Research  Council,  2004).  Different 
organizations  have  different  perceptions  on  how  business  gets  done.  They  have  different 
goals  and  timelines,  with  different  chains  of  command.  All  of  these  cultural  differences 
are  in  some  way  possible  barriers  to  technology  transition.  The  National  Research 
Council  proposes  that  the  introduction  and  acceptance  of  new  technologies  often  depend 
more  on  social,  cultural,  and  historical  factors  than  on  technological  merit. 

The  laboratories  tend  to  have  the  perception  that  the  role  of  the  S&T  community 
is  technology  pull;  meaning  if  it  is  a  good  technology  they  (the  customer)  will  come 
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(Defense  Systems  Acquisition  Management  Course,  2004).  Technology  pull  means  the 
user  is  asking  for  specific  capabilities,  in  which  they  seek  out  an  organization  such  as 
AFRL  to  provide  that  capability.  On  the  other  hand,  technology  push  can  be  defined  as 
materials  that  are  marketed  to  systems  engineers  and  designers,  typically  in  materials  and 
processes  organizations,  with  the  goal  of  lining  up  funding  commitments  (National 
Research  Council,  2004).  If  both  organizations  can  work  together,  then  the  focus  shifts 
from  technology  push/pull  to  a  work  environment  that  fosters  collaboration  between  the 
S&T  community  and  the  acquisition  community.  Other  cultural  perceptions  include 
ideas  such  as:  the  job  is  complete  at  the  technology  development  stage;  implementation 
of  the  technology  is  the  customers  responsibility;  the  S&T  community  only  pushes 
technology;  the  development  cycle  for  S&T  products  is  too  long  for  the  acquisition  and 
user  community;  and  the  laboratories  focus  only  on  the  technology  side  of  the  project  and 
leave  the  business  part  to  other  organizations  (Defense  Systems  Acquisition  Management 
Course,  2004).  These  cultural  barriers  usually  exist  because  of  poor  communication, 
absent  transition  strategies,  and  lack  of  incentives  (Manager’s  Guide  to  Technology 
Transition  in  an  Evolutionary  Acquisition  Environment,  2003). 

2.4  PROGRAMS  THAT  FACILITATE  TECHNOLOGY  TRANSITION 

It  is  important  to  note  that  there  are  many  programs  in  existence  that  facilitate 
technology  transition.  In  this  study,  I  will  discuss  programs  pertinent  to  my  thesis,  with 
emphasis  on  AFRL  and  the  acquisition  community.  Programs  that  do  facilitate  the 
transition  of  technologies  but  will  not  be  focused  on  are:  Dual  Use  Science  and 
Technology  (DUS&T);  Manufacturing  Technology  (ManTech);  and  the  Small  Business 
Innovative  Research  (SBIR)  program.  These  programs  will  not  be  covered  because  they 
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are  typically  jointly  funded  by  different  organizations  such  industry  and  the  Office  of  the 
Secretary  of  Defense  (OSD).  Programs  that  are  jointly  funded  have  different  processes 
and  regulations  than  programs  such  as  Advanced  Technology  Demonstrations  (ATDs). 

Programs  that  will  be  addressed  that  facilitate  technology  transition  are: 

Advanced  Technology  Demonstration  (ATD);  Advanced  Concept  Technology 
Demonstration  (ACTD);  and  the  Technology  Transition  Initiative  (TTI).  These  programs 
will  contribute  to  this  study.  Technologies  available  for  transition  usually  come  from 
either  the  ATD  process  or  an  ACTD  program  (Dobbins,  2004).  These  programs  will  be 
discussed  in  detail  for  the  reason  that  they  are  significant  in  the  way  AFRL  can  transition 
technologies  to  the  warfighter. 

The  first  program  that  will  be  discussed  is  the  ATD  program.  ATDs  are  a  process 
for  managing  S&T  programs  that  brings  the  team  together  early,  and  demonstrates  a 
military  capability  in  a  joint  warfighting  experiment,  battle  lab  experiment, 
demonstration,  field  test,  or  simulation  (Manager’s  Guide  to  Technology  Transition  in  an 
Evolutionary  Acquisition  Environment  2003).  The  main  focus  of  an  ATD  is  to  transition 
a  mature  technology  to  the  warfighter  that  helps  improve  a  legacy  system,  or  provides  a 
new  capability  that  the  warfighter  can  use  to  improve  operational  capabilities. 

Typically,  ATDs  procure  an  agreement  between  the  research  laboratories  and  the 
customer  through  transition  plans  and  exit  strategies.  However,  ATDs  are  usually 
prioritized  by  the  MAJCOMs,  which  signify  that  the  MAJCOMs  have  the  final  say  in 
what  becomes  an  ATD.  If  a  6.3  project  does  not  become  an  ATD  then  it  is  classified  as  a 
Critical  Experiment  (CE).  CEs  should  still  follow  the  same  transition  path  as  ATDs.  The 
principal  difference  between  an  ATD  and  a  CE  is  the  ATD  has  more  funding  allotted  for 
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demonstrations,  while  CEs  have  very  little  funding  allocated  for  demonstrations.  These 
demonstrations  can  be  very  important  for  two  reasons:  ( 1 .)  it  provides  another  avenue  to 
test  the  product;  (2.)  it  provides  an  opportunity  for  the  PM  to  show  their  technology  to 
senior  leaders.  This  is  another  substantial  reason  as  to  why  the  S&T  PM  must  involve 
other  organizations,  such  as  the  MAJCOMs,  early  in  the  6.3  program.  The  ATC  will 
divide  the  ATDs  in  different  categories,  which  are  found  in  Table  2. 

Table  2  ATD  Categories 


Category  1 

MAJCOM  or  Agency  supports  and  has 
programmed  required  funding  for  transition 
within  the  FYDP 

Category  2A 

MAJCOM  or  Agency  supports  and  is 
committed  to  identify  transition  funding  in  the 
next  Program  Objective  Memorandum  (POM) 
cycle  or  Amended  POM 

Category  2B 

MAJCOM  or  Agency  supports  but  is  not 
currently  able  to  program  for  transition  funding 

(AFI  61-101) 

Once  there  is  agreement  between  the  labs  and  the  MAJCOMs,  it  does  not  mean  that  the 
technology  will  transition.  Before  the  MAJCOM  takes  responsibility  for  the  technology, 
roadblocks  such  as  funding,  contracting  strategy,  and  interoperability  can  still  prevent 
successful  completion  of  the  process. 

ATDs  require  transition  plans  just  like  many  other  projects  that  plan  on 
transitioning  from  the  laboratory  to  another  entity.  ATDs  are  not  funded  by  an  outside 
source  and  require  all  programs  going  through  to  have  their  own  transition  plans,  exit 
criteria,  and  schedules.  Agencies,  such  as  AFRL,  have  their  own  processes  for 
nominating  specific  projects  to  become  an  ATD.  Some  criteria  that  a  technology  must 
meet  in  order  to  become  an  ATD  are:  the  new  concept  addresses  S&T  objectives; 
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enhances  military  capability  or  provides  a  new  capability;  has  a  cost-effective  approach; 
and  is  funded  with  completion  time  being  less  than  five  years  (The  Manager’s  Guide  to 
Technology  Transition  in  an  Evolutionary  Acquisition).  For  a  project  to  be  nominated  to 
an  ATD  it  must  seek  approval  through  the  Applied  Technology  Council  (ATC). 

The  ATC  must  approve  each  transition  plan  for  the  ATD  candidate.  The  ATC  is 
made  up  of  senior-level  management  from  the  MAJCOMS,  Product  Centers,  and 
representatives  from  AFRL.  The  members  that  specifically  fonn  the  ATC  are:  the  Chair 
(HQ  AFMC  Vice  Commander  for  AFMC  ATC,  HQ  AFSPC  Vice  Commander  for 
AFSPC  ATC,  and  AFMC  Fead  Enterprise  Commander),  Primary  Members  (MAJCOM 
Vice  Commanders,  Agency  Commanders,  AFRF  Commander,  and  Air  Force  Acquisition 
Product  and  Fogistic  Center  Commanders),  Secretariat  (usually  designated  by  the  Fead 
Enterprise  Commander),  and  Invited  Participants  (usually  AETC  and  other  senior  level 
stakeholders  or  advisory  representatives)  (AFI  61-101). 

The  purpose  of  the  ATC  is  to  ensure  technologies  are  transitioned  to  the 
acquisition  community  in  a  timely  manner.  The  ATC  was  also  established  so  the 
acquisition  communities  (MAJCOMs  and  program  offices)  have  a  better  understanding 
of  what  is  going  on  in  the  S&T  community.  The  main  goal  of  the  ATC  is  to  facilitate  the 
transition  of  ATD  programs  to  the  Air  Force  Material  Command  (AFMC),  Air  Force 
Space  Command  (SPACECOM),  and  other  MAJCOMs  (AFI  61-101). 

ACTDs  are  another  possible  avenue  used  to  transition  lab  projects  to  the 
warfighter.  The  ACTD  program  was  initiated  by  the  DOD  in  1994  as  a  way  to  get  new 
technologies  that  meet  critical  military  needs  into  the  hands  of  users  faster  and  at  less  cost 
than  the  traditional  acquisition  process  (GAO,  2002).  ACTDs  are  usually  a  new 
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technology  or  concept  that  users  are  not  very  familiar  with.  It  is  also  important  to  note 
the  difference  between  ATD  and  ACTD.  ACTDs  are  programs,  usually  employing 
multiple  technologies  that  are  reviewed  by  OSD  and  the  joint  requirements  oversight 
council  (JROC),  and  partly  funded  with  OSD  ACTD  funds  (Manager’s  Guide  to 
Technology  Transition  in  an  Evolutionary  Acquisition  Environment  2003).  An  ATD  is  a 
process  not  a  program;  however,  the  biggest  difference  is  that  ACTDs  are  partly  funded 
by  another  organization  and  ATDs  are  not.  Another  major  difference  is  that  ATDs  are 
reviewed  by  the  ATC  and  ACTDs  are  reviewed  at  the  OSD  level.  ATDs  are  reviewed 
and  approved  by  the  services  (MAJCOMs)  and  funded  with  service  S&T  funds 
(Manager’s  Guide  to  Technology  Transition  in  an  Evolutionary  Acquisition  Environment 
2003). 

A  key  goal  of  ACTDs  is  to  move  into  the  appropriate  phase  of  formal  acquisition 
without  loss  of  momentum,  assuming  the  user  makes  a  positive  determination  of  military 
utility  (http://www.acq.osd.mil/actd/intro.htm).  ACTDs  are  managed  by  the  user  or 
sponsor.  For  example,  once  AFRL  transitions  a  technology  to  a  gaining  unit  or 
warfighter,  the  user  would  assume  oversight  responsibilities  for  that  technology  including 
funding  and  PM  responsibilities.  Another  primary  aspect  of  ACTDs  is  that  the  program 
exploits  mature  technologies.  Just  as  ATDs  should  be  mature  technologies  ACTDs  must 
be  able  to  demonstrate  capabilities  at  TRL  levels:  6  -  System/subsystem  model  or 
prototype  demonstration  in  a  relevant  environment,  7  -  System  prototype  demonstration 
in  an  operational  environment,  8  -  Actual  system  completed  and  operationally  qualified 
through  test  and  demonstration  (ground  or  space),  9  -  Actual  system  operationally  prove 
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through  successful  mission  operations)  before  technologies  can  transition  to  an  ACTD 
(Dobbins,  2004). 

A  fairly  new  program  is  the  Technology  Transition  Initiative  (TTI)  program.  Just 
like  ATDs  and  ACTDs,  the  TTI  program  is  focused  on  facilitating  the  rapid  transition  of 
new  technologies  from  S&T  programs  into  acquisition  programs  (Technology  Transition 
Initiative  Proposal  Template,  Fiscal  Year  2006).  The  recurring  objective  for  programs 
such  as  ATD,  ACTD,  and  TTI  is  that  these  programs  were  put  in  place  to  get  advanced 
technologies  over  the  valley  of  death  and  into  the  hands  of  the  warfighter.  The  TTI  was 
developed  by  the  office  Under  Secretary  of  Defense  (Acquisition,  Technology,  & 
Logistics)  (USD/AT&L)  so  promising  technological  innovations  could  transition  to  the 
operational  communities  (Technology  Transition  Initiative  Proposal  Template,  Fiscal 
Year  2006). 

Projects  that  are  selected  for  TTI  funds  are  reviewed  at  the  OSD  level.  One  key 
benefit  of  the  TTI  is  that  the  projects  selected  for  this  program  receive  50%  of  its  funding 
with  OSD  money.  This  influx  of  dollars  alleviates  some  of  the  pressure  on  the  PMs  and 
MAJCOMs  when  funding  issues  arise.  The  main  difference  between  the  TTI  program 
and  ATD  programs  are  the  different  types  of  funding.  ATDs  typically  do  not  receive  any 
funding  from  other  organizations  until  the  technology  transitions  or  is  about  to  transition. 
Funding  has  been  established  to  be  a  critical  factor  if  technologies  are  to  stay  out  of  the 
valley  of  death.  Projects  that  are  selected  through  TTI  will  help  spark  the  interest  in 
funding  among  all  the  key  stakeholders. 

Throughout  this  chapter  I  have  discussed  roadblocks  to  technology  transition, 
programs  that  facilitate  technology  transition,  technology  transition  best  practices,  and 
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guidelines  that  organizations  need  to  consider  when  dealing  with  the  process  of 
technology  transition.  As  stated  in  Chapter  One,  there  has  not  been  an  intense  focus  on 
AFRL  and  technology  transition.  By  leveraging  government  sponsored  documents,  Air 
Force  Directives,  and  industry  practices  I  have  analyzed  and  extracted  nine  commonly 
used  best  practices  for  technology  transition.  These  best  practices  will  orchestrate  the 
collection  of  data  necessary  for  this  case  study  through  structured  interviews.  Use  of  the 
nine  best  practices  formulates  the  dynamics  of  this  study  to  examine  the  extent  to  which 
AFRL  currently  employs  known  technology  transition  practices. 
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II.  METHODOLOGY 


3.1  OVERVIEW 

This  chapter  will  discuss  the  methodology  chosen  to  conduct  this  research,  as  well 
as  techniques  used  for  data  collection.  As  Spivey  (1997)  discussed,  there  has  been  little 
evidence  published  on  technology  transition  within  DOD  laboratories.  For  this  reason 
alone  an  exploratory  approach  with  a  case  study  design  was  determined  to  be  the  most 
suitable  approach.  The  primary  objective  of  this  case  study  is  to  capture  the 
circumstances  and  conditions  of  an  everyday,  or  commonplace,  situation  while  assuming 
the  lessons  learned  from  these  cases  will  be  informative  as  to  the  experiences  of  the 
average  person  or  institution  (Yin,  2003).  The  purpose  with  this  approach  (Yin,  2003  and 
Creswell,  1994)  is  to  gain  insight  into  how  the  program  manager  views  and  practices 
technology  transition  guidance.  The  four  sections  in  this  chapter  will  discuss:  research 
methodology,  qualitative  design  approach,  the  data  analysis  model,  and  ensuring  research 
validity. 

3.2  QUALITATIVE  RESEARCH  METHOD 

The  components  used  to  structure  this  research  design  were:  study  questions;  unit 
of  analysis;  and  criteria  for  interpreting  data  (Yin,  2003).  The  research  question  for  this 
study  was: 

To  what  extent  is  AFRL  implementing  known  technology  transition  best  practices 
from  industry  and  DOD  guidance  to  accelerate  the  transition  of  laboratory 
technologies  to  the  warfighter? 

Yin  (2003)  and  Creswell  (1994)  agree  that  the  essence  of  a  case  study  design  is  to 
investigate  a  phenomenon  in  real  life  context,  especially  when  boundaries  are  not  clearly 
defined.  Technology  transition  involves  many  different  organizations  and  possibly 
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multiple  processes,  which  makes  the  boundaries  unclear.  After  conducting  the  literature 
review  it  was  apparent  that  guidance  on  technology  transition  is  not  an  exact  science. 

The  investigative  questions  are: 

1 .  What  guidance  exists  for  technology  transition  within  the  DOD? 

2.  What  is/are  the  process  or  processes  that  exist(s)  to  transition  technologies  to  the 
warfighter? 

3.  Do  the  managers  in  the  S&T  community  believe  the  current  guidance  is  adequate 
to  effectively  transition  technologies  to  the  warfighter? 

4.  Is  there  best  practices  not  being  currently  implemented  in  AFRL  that  could 
enhance  the  transition  of  laboratory  technologies? 

The  research  question  and  investigative  questions  attempt  to  gauge  how 
technologies  are  transitioned,  if  best  practices  are  being  used,  what  are  the  problems,  and 
if  there  are  problems,  how  these  issues  could  be  addressed.  These  viewpoints  will  be 
based  on  those  opinions  of  the  program  managers  interviewed.  Since  the  study  was 
centered  on  one  topic,  technology  transition,  it  led  to  the  rationale  of  a  single  case  study 
approach.  With  a  focus  on  technology  transition  within  the  Air  Force,  AFRL  was  chosen 
to  be  the  unit  of  analysis.  This  posed  difficulty  since  technology  transition  has  no  bounds 
and  can  involve  many  different  organizations  (Yin,  2003). 

In  order  to  gather  data,  a  focused  interview  was  used  (Merton  et  al,  1990).  A 
focused  interview  best  serves  the  purpose  of  the  study  because  the  nature  of  the  research 
revolves  around  nine  technology  transition  best  practices.  The  interview  focused  on 
these  best  practices  and  followed  up  with  some  general  overarching  questions.  The 
interviews  also  focused  on  a  phenomenon  of  “human  affairs”  (Yin,  2003),  since  the 
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nature  of  technology  transition  can  involve  many  communities  with  different  cultures  and 
perceptions. 

A  Subject  Matter  Expert  (SME)  within  AFRL  was  asked  to  provide  a  list  of  PMs 
who  had  experience  in  transitioning  laboratory  projects  to  the  warfighter.  The  SME 
provided  a  list  of  program  managers  within  AFRL  that  had  experience  working  with  6.3 
projects.  Once  the  interview  questions  were  developed  they  were  reviewed  by  SMEs  and 
colleagues  within  the  organization.  Redundant  questions  were  removed  and  valid 
questions  were  added  per  SME  request.  Interviews  were  broken  down  to  statements 
after  they  were  transcribed.  Statements  were  analyzed  for  common  themes.  The 
interviews  were  designed  to  specifically  measure  AFRL  technology  transition  practices 
as  well  as  thoughts  and  viewpoints  on  the  process.  With  this  approach,  open-ended 
questions  were  chosen  to  make  up  the  interview  sample  (Creswell,  1994,  Yin,  2003). 

After  interview  candidates  were  identified,  they  were  contacted  via  email  and 
phone  calls.  Most  interviews  were  scheduled  in  advance.  All  interviews  were  conducted 
face  to  face  at  the  offices  of  the  participating  interviewees.  Notes  were  taken  during  each 
interview  and  recorded  when  the  participant  gave  permission.  It  was  also  explained  that 
all  interviews  would  remain  confidential  and  the  participants  would  be  unidentifiable. 

The  interview  questions  focused  on  the  subjects’  views  about:  a)  technology 
transition  processes;  b)  current  technology  transition  guidance  and  the  effectiveness  it  has 
on  transitioning  technologies  to  the  warfighter;  c)  processes  currently  being  used  to 
transition  technologies  to  the  warfighter;  d)  their  role  in  implementing  technology 
transition.  The  AFRL  Internal  Review  Board  granted  permission  to  conduct  these 
interviews.  Interview  questions  are  provided  in  Table  3  below. 
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Table  3  Interview  Questions 


BEST  PRACTICES 

QUESTION 

Information  Technology  (IT) 

Is  your  project  currently  identified/described  in  a  sharable  database  in  which  your  customer  has 
access  to? 

Do  you  have  the  proper  tools  for  transitioning  your  project? 

Do  you  have  problems  with  software  interoperability?  For  example,  can  you  easily  send  your 
customer  a  large  file,  do  you  have  the  software  tools  to  open  and  modify  their  schedules,  etc.? 

Do  you  know  what  your  customers  IT  tool  set  includes? 

Are  you  aware  of  the  Virtual  Technology  Exposition  (VTE)  website  which  is  a  database  that  shares 
information  about  ongoing  R&D  products  and  acquisition  programs? 

Communication 

How  do  you  communicate  with  your  customer?  i.e.  email/phone/meetings 

How  often? 

What  kind  of  relationship  do  you  have  with  your  customer? 

What  are  your  communication  barriers? 

Does  your  customer  fully  understand  what  you  are  delivering? 

Do  you  do  anything  jointly  with  your  customer  (write  concept  papers,  attend  conferences,  joint 
briefings,  etc.) 

Identify  the  Customer 

Do  you  know  who  your  customer  is? 

Is  your  effort  a  technology  push  or  technology  pull  effort? 

Was  the  customer  identified  early  on  your  project? 

Participation 

Does  you  customer  actively  participate  on  a  regular  basis? 

Do  necessary  acquisition  personnel  participate  when  needed? 

How  would  you  describe  your  relationship  with  the  acquisition  community? 

Could  it  be  better? 

What  type  of  interaction  do  you  have  with  industry? 

Do  you  have  any  documents  that  outline  your  customer’s  roles  and  responsibilities  and  how  they 
should  be  participating  on  the  effort? 

If  you  have  an  outstanding  relationship  with  your  customer  what  would  you  attribute  to  that  strong 
relationship? 

Knowledge  Management 

Would  you  say  that  the  majority  of  the  PMs  in  AFRL  are  familiar  with  acquisition  processes?  i.e. 
Schedule,  POM,  PMRs,  requirements  generation,  financial  management,  defense  acquisition  etc 

Do  you  know  early  on  or  even  at  all  if  your  project  can  be  transitioned/inserted  into  a  legacy 
system? 

Do  you  base  your  technology  availability  dates  on  weapon  system  transition  windows? 

Do  you  understand  how  your  technology  fits  in  providing  new  warfighter  capabilities? 

Can  you  very  clearly  define  the  As-Is  environment  and  then  describe  the  environment  with  your 
technologies  implemented? 
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Incentives 


What  incentives  do  scientists  or  PMs  on  a  product  development  team  have  to  reach  out  to  the 
customer? 

What  incentives  do  you  have  to  transition  your  project? 

Do  you  have  any  suggestions  on  possible  incentives? 

Affordability  Metrics 

Do  you  use  metrics  for  your  project  schedule  and  performance  criteria? 

How  do  you  determine  your  exit  criteria? 

Do  you  have  the  right  personnel  to  measure  these  aspects  of  the  project? 

Did  you  define  or  coordinate  your  key  performance  parameters  with  your  customer? 

Did  you  perform  an  Return  on  Investment  (ROI)  or  create  a  business  case  at  the  start  of  your 
technical  efforts? 

Technology  Transition  (TT)  Team 

Do  you  think  a  technology  transition  team  would  help  market  AFRL  projects? 

Do  you  think  PMs  in  the  labs  have  the  right  skills  to  do  PM  work? 

Do  the  product  development  teams  have  the  right  expertise/personnel  to  facilitate  TT? 

Are  there  other  career  fields  that  could  be  of  benefit  to  help  projects  transition? 

Would  you  be  in  favor  of  having  someone  else  work  the  transition  issues  on  your  program? 

Technology  Maturity 

How  do  you  ensure  technology  maturity  for  your  project? 

Are  the  Technology  Readiness  Level  (TRLs)  an  appropriate  gauge  to  measure  technology  maturity? 
Do  you  track  and  assess  technology  maturation  by  key  milestones  or  just  prepare  a  chart  for 
Program  Baseline  Reviews? 

Do  tire  TRLs  help  you  in  communicating  with  your  customer  and  in  meeting  their  expectations? 

General  Questions 

In  your  opinion  what  do  you  think  the  single  most  important  roadblock  in  transitioning  AFRL 
projects? 

Do  you  think  AFRL  representation  in  the  program  offices  could  help  transition  AFRL  projects? 
What  do  you  think  the  process  is  that  sells  your  project;  i.e.  transitions  it  or  markets  it? 

Who  is  your  face  to  push  your  project  to  be  transitioned  (i.e.  the  PM,  Applied  Technology  Council 
(ATC),  Acquisition  community)? 

Do  you  think  the  ATC  does  a  good  job  to  ensure  technologies  are  transitioned? 

Do  you  think  the  Applied  Technology  Council  process  works  to  prevent  delays  in  transition? 

How  do  you  go  about  getting  a  champion  for  your  project? 

What  organization  is  the  most  difficult  to  get  buy  in  from? 


3.3  DATA  ANALYSIS  TECHNIQUE 

The  qualitative  data  collected  through  this  case  study  used  Creswell’s  (2003) 
analysis  spiral  as  a  template.  The  data  collected  was  reviewed  on  multiple  occasions 
using  the  following  guidelines: 
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1 .  Organization:  Data  was  collected  via  digital  recorder  when  the  researcher 
obtained  pennission.  The  data  was  then  transcribed,  broken  up  into  smaller  phrases, 
and  then  stored  in  a  database. 

2.  Perusal:  Overarching  concepts  were  reviewed  multiple  times.  Ideas  and  concepts 
were  then  broken  down  to  an  understanding  of  the  underlying  meaning. 

3.  Classification:  Themes  and  topics  were  extracted  to  make  up  a  list  of  common 
categories. 

4:  Synthesis:  All  data  was  summarized.  Propositions  and  hypotheses  emerged  which 
identified  categorical  relationships. 

Data  analysis  techniques  included  pattern  matching,  explanation  building,  and 
theme  development  (Trochim,  1989,  Creswell,  1994,  and  Yin,  2003).  After  using  the 
spiral  approach  above,  patterns  and  themes  emerged  after  synthesizing  the  data.  This 
approach,  compared  to  predicted  patterns,  strengthened  the  internal  validity  (Trochim, 
1989).  Explanation  building  was  used  to  refine  a  set  of  ideas  and  link  them  to  plausible 
or  rival  explanations  (Yin,  2003).  Another  purpose  for  explanation  building  is  to 
generate  concepts  for  future  study.  Safeguards  used  to  reduce  problems  associated  with 
explanation  building  constantly  referenced  the  reason  for  the  inquiry  and  third  party 
inspection  of  raw  data  during  the  data  collection  process.  After  data  analysis  was 
conducted  the  next  step  was  to  ensure  the  validity  of  the  research. 

3.4  ENSURING  VALIDITY  AND  RELIABILITY 

To  validate  the  research  employed,  the  researcher  must  clearly  state  the  bias  they 
bring  to  the  study  (Creswell,  1994).  The  researcher  in  this  case  did  have  past  experience 
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working  with  AFRL.  To  control  for  bias,  participants  were  randomly  selected.  The  only 
criteria  the  participants  had  to  meet  were  they  needed  be  a  6.3  program  manager. 

Peer  input  was  the  second  method  used  to  validate  the  research.  Having 
colleagues  and  instructors  review  the  data  in  draft  form  ensured  construct  validity  (Yin, 
2003).  Subject  Matter  Experts  within  AFRL  also  had  the  opportunity  to  review  the 
interview  questions  and  research  questions.  This  ensured  explanation  for  this  research 
was  clear  and  concise  for  other  readers.  Internal  feedback  was  addressed  through  the  use 
of  this  method. 

Throughout  this  analysis,  recurrent  themes  and  patterns  emerged.  Pattern 
matching  was  used  to  interpret  the  data  due  to  the  lack  of  quantitative  data.  Pattern 
theory  uses  analogies  to  tie  relationships  and  ideas  together  (Neuman,  1991).  These 
themes  and  patterns  along  with  statements  from  the  interviewees  were  given  to  co¬ 
workers  for  review.  These  individuals  were  asked  to  categorize  the  statements  under  the 
themes.  The  purpose  for  this  was  to  see  if  they  drew  the  same  conclusions  as  the 
researcher.  The  external  audit  by  those  not  associated  with  this  study  validates  the 
synthesis  of  the  data. 

All  data  collection  and  analysis  procedures  were  documented  to  ensure  reliability. 
The  goal  was  to  minimize  as  many  errors  as  possible  (Yin,  2003).  Case  study  skills 
suggested  by  Yin  (2003)  were  employed  throughout  data  collection,  and  analysis 
ensured:  reliability  was  at  the  beginning  gain  a  firm  understanding  of  the  issues  being 
studied,  the  researcher  checked  all  perceptions  “at  the  door”  before  each  interview  for  a 
variety  of  reasons:  to  be  more  responsive  to  the  data;  allowed  the  questions  to  be  flexible 
and  adaptive;  which  increased  opportunities  for  new  insight  that  they  had  not  been  privy 
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to;  which  ensured  good  questions  were  asked  by  having  them  reviewed  by  SMEs.  These 
data  collection  methods  and  analysis  procedures  allowed  the  researcher  to  gain  a  great 
deal  of  understanding  and  insight  into  the  phenomena  technology  transition. 
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IV.  DATA  ANALYSIS 


4.1  INTRODUCTION 

The  focus  of  this  effort  is  to  evaluate  the  extent  AFRL  is  implementing  known 
technology  transition  best  practices  from  industry  and  DOD  standards  in  order  to 
accelerate  the  transition  of  laboratory  technologies  to  the  warfighter.  Through  review  of 
the  literature  nine  best  practices  became  the  baseline  for  the  interview  questions.  Then 
interviews  were  conducted  with  S&T  program  managers  that  had  experience  managing 
6.3  programs.  This  chapter  will  discuss  data  captured  through  interviews  and  subsequent 
analysis.  Patterns  and  themes  were  discovered  using  qualitative  analysis  techniques.  The 
following  sections  will  discuss  the  analysis  of  the  data  while  pointing  out  themes  that 
occurred  for  each  technology  transition  best  practice. 

4.2  DATA  ANALYSIS 

Best  Practice  One:  Information  Technology 

(Q 1-1)  Is  your  project  currently  identified  or  described  in  a  sharable  database  in 

which  your  customer  has  access  to? 

It  is  understood  that  there  are  many  IT  tools  out  there.  Even  though  this  question 
focused  on  shareable  databases,  the  purpose  for  this  question  was  to  get  the  participant 
thinking  about  how  they  use  IT  tools  to  collaborate  with  their  customers.  Four  out  of  the 
twenty-three  interviewed  did  not  share  data  through  the  use  of  a  sharable  database.  Three 
individuals  suggested  that  the  tools  exist,  but  do  not  use  them  because  no  one  wants  the 
responsibility  to  maintain  the  database.  However,  nineteen  out  the  twenty-three  subjects 
saw  value  in  leveraging  a  sharable  database  to  conduct  project  business  with  their 
customer.  The  individuals  that  did  say  they  leveraged  sharable  databases  stated  they 
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accomplished  this  through  web-based  tools  or  a  database  that  was  managed  by  their 
contractor. 

(Ql-2)  Do  you/customer  have  the  proper  tools  for  communicating  project  issues? 
Twenty  out  of  twenty-three  subjects  thought  they  had  the  proper  IT  tools  to 
effectively  communicate  and  share  data  with  their  customer.  Three  out  of  the  twenty- 
three  interviewed  stated  they  did  not  have  the  proper  tools  needed  to  effectively 
communicate  with  their  customer.  One  subject  felt  an  instant  messaging  capability  was 
needed. 

(Ql-3)  Do  you  have  problems  with  software  interoperability?  For  example,  can 
you  easily  send  your  customer  a  large  file,  do  you  have  the  software  tools  to  open 
and  modify  their  schedules,  etc.  ? 

Three  out  of  twenty-three  did  not  feel  they  had  any  issues  with  interoperability. 
Twenty  individuals  said  they  did  have  these  issues  on  a  regular  basis.  Issues  ranged  from 
access  to  classified  data  and  secure  applications,  organizational  barriers  (i.e.  access  to 
software  applications  that  are  stovepiped),  and  contracting  issues.  It  is  important  to  note 
that  reasons  such  as  access  to  classified  data  and  secure  applications  came  from  one 
directorate. 

(Ql-4)  Do  you  know  what  your  customers  IT  tool  set  includes? 

Ah  twenty-three  participants  understand  what  IT  tools  the  customer  uses.  Two 

participants  noted  that  that  the  tools  exist  but  are  not  leveraged  enough  as  they  should. 

The  participants  stated  that  all  the  customers  typically  did  not  have  a  specific  set  of  tools, 

but  did  have  access  to  tools  needed  to  collaborate  with  the  stakeholders. 

(Q 1  -5)  Are  you  aware  of  the  Virtual  Technology  Exposition  (VTE)  website  which 
is  a  database  that  shares  information  about  ongoing  R&D  products  and 
acquisition  programs? 
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The  purpose  for  this  question  was  to  get  the  participant  to  think  about  the 
usefulness  of  tools  like  the  VTE.  One  out  of  twenty-three  interviewed  did  know  about 
the  website.  Five  individuals  did  not  have  an  answer.  Seventeen  out  of  twenty-three 
interviewed  had  never  heard  of  this  website.  Even  though  the  majority  of  participants 
had  never  heard  of  the  VTE,  they  all  could  see  some  utility  in  a  tool  like  this. 

Two  themes  did  emerge  from  this  section.  One  was  the  importance  participants 
placed  on  the  use  of  IT  tools.  Table  4  illustrates  the  results.  Of  those  individuals 
interviewed,  90%  had  a  positive  response  about  IT  tools  and  used  them  to  reach  out  to 
their  customer  and  share  data.  Questions  Q 1  - 1 ,  Ql-2,  and  Ql-4  were  combined  to  gain  a 
more  accurate  point  of  view  on  IT  from  the  perspectives  of  the  PMs.  Theme  one 
emerged  using  69  responses.  The  program  managers  interviewed  stated  they  leveraged 
tools  such  as  web-based  tools  and  shared  databases.  They  all  used  these  IT  tools  to 
conduct  business  on  a  regular  basis.  Even  though  95%  of  those  interviewed  were  not 
aware  of  the  VTE  tool  that  was  illustrated  in  the  Technology  Transition  for  Affordability: 
A  Guide  for  S&T  Program  Managers,  they  were  aware  of  web-based  tools  that  provided 
similar  functionality. 

Another  theme  that  emerged  was  the  overwhelming  response  of  issues  dealing 
with  IT  tools  and  interfaces.  Theme  two  was  considered  from  23  responses.  Of  the 
twenty-three  individuals  interviewed,  87%  had  issues  with  their  IT  infrastructure. 
However,  even  though  87%  had  issues  with  their  IT  tools,  they  still  saw  value  in  the  use 
of  IT  tools  when  working  technology  transition  issues. 
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Table  4  Usefulness  of  IT  Tools 


Themes 

#  of  responses 

%  of  participants 

#  of  participants 

1 .  Use  of  IT  tools 

(out  of  69) 

Positive  Statement 

64 

93% 

23 

Negative  Statement 

5 

7% 

23 

2.  Experienced  IT  problems 

(out  of  23) 

20 

87% 

23 

Had  no  IT  problems 

3 

13% 

23 

Best  Practice  Two:  Communication 

(Q2-1)  How  do  you  communicate  with  your  customer? 

The  majority  of  the  participants  used  email  when  communicating  with  their 
customer.  Eleven  of  the  twenty-three  participants  typically  communicated  with  their 
customer  through  email.  Seven  responded  as  using  email,  phone,  and  meetings.  Two 
respondents  answered  telephone,  while  another  two  answered  face  to  face.  One  subject 
responded  with  the  answer  telecoms. 

(Q2-2)  How  often  do  you  communicate  with  your  customer? 

It  seemed  all  the  participants  communicated  often  with  their  customer.  Thirteen 
subjects  said  they  communicate  on  a  weekly  basis  with  their  customer.  Two  responded 
as  communicating  on  a  monthly  basis  with  their  customer,  while  another  two  said  on  a 
quarterly  basis.  Six  respondents  communicated  on  a  daily  basis  with  their  customer. 

(Q2-3)  What  kind  of  relationship  do  you  have  with  your  customer? 

Fifteen  respondents  believed  they  had  a  good  relationship  with  their  customer. 
Eight  of  the  interviewees  said  they  just  had  an  OK  relationship  with  their  customer.  The 
subjects  that  described  their  relationship  with  the  customer  as  OK  generally  agreed  it  was 
because  of  personality  differences  or  the  busy  schedules  of  the  customer. 
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(Q2-4)  What  are  your  communication  barriers? 

Four  respondents  said  they  had  no  communication  barriers  they  were  aware  of. 
Five  agreed  that  turnover,  especially  at  the  MAJCOM  and  Program  Office  level,  really 
hindered  the  communication  process.  Five  out  of  the  twenty-three  interviewed  thought 
personality  issues  were  the  reasons  for  communication  breakdowns.  Another  set  of  five 
respondents  attributed  the  breakdown  in  communication  to  a  lack  of  understanding  from 
the  different  organizations  involved  (MAJCOMs,  Program  Offices,  and  Laboratories).  It 
is  important  to  note  that  the  lack  of  understanding,  whether  it  is  the  conveyance  of 
organizational  goals  or  technical  knowledge  by  the  customer,  was  apparent  throughout 
multiple  questions.  One  answered  that  distance  between  them  and  the  customer  posed  a 
problem  in  communication.  Another  responded  that  the  access  to  classified  tools  such  as 
Secure  Telephone  Equipment  (STE)  and  Secure  Terminal  Unit  (STU),  limited  their 
ability  to  communicate  with  the  customer.  One  individual  stated  the  inability  to  have 
senior  leaders  talking  to  leadership  in  other  organizations  was  a  barrier  to 
communication.  The  last  individual  interviewed  stated  firewalls  as  a  limiting  factor  in 
communicating  with  their  customer. 

(Q2-5)  Does  your  customer  fully  understand  what  you  are  delivering? 

The  majority  of  the  participants  attributed  their  customers  understanding  to  good 
communication  and  involvement.  Eighteen  out  of  the  twenty-three  respondents  believed 
their  customer  fully  understood  the  technology  capability  they  were  delivering.  The 
other  five  subjects  did  not  believe  their  customers  understood  the  technology  they  were 
receiving.  Four  out  of  the  five  respondents  that  stated  no  believed  the  customer  only 
understood  the  novelty  of  the  technology,  but  not  the  real  “in  and  outs.” 
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(Q2-6)  Do  you  do  anything  jointly  with  your  customer?  (write  concept  papers, 

attend  conferences,  joint  briefings,  etc.) 

The  purpose  for  this  question  was  to  gain  insight  on  the  collaboration  between  the 
PM  and  the  customer.  Fifteen  out  of  the  twenty-three  respondents  perfonned  in  joint 
efforts  such  as  concept  papers,  joint  briefings,  and  attending  conferences  with  their 
customers.  Eight  individuals  did  not  work  joint  efforts  with  their  customer.  Again  the 
eight  individuals  that  did  not  conduct  joint  efforts  such  as  these  attributed  it  to  the  busy 
schedules  of  the  customers. 

A  couple  of  themes  did  emerge  from  this  section  of  questions.  Questions  2-1,2- 
2,  2-3,  2-4,  and  2-6  combined  to  get  an  overall  understanding  of  how  effective  the 
communication  was  between  the  PM  and  the  customer.  These  questions  combined 
totaled  115  responses.  The  responses  for  themes  one  and  two  can  be  found  in  Table  5. 
Out  of  these  five  questions  dealing  with  communication  87%  had  a  positive  response. 
Program  managers  typically  performed  joint  efforts  with  their  customer.  Most  of  the 
PMs  could  say  they  were  confident  they  had  communicated  enough  to  the  customer  so 
they  could  understand  what  technological  capabilities  they  were  receiving. 

The  second  theme  that  emerged  was  communication  issues.  Theme  two  emerged 
from  23  responses.  All  twenty-three  PMs  stated  they  had  good  relationships  with  their 
customer,  however,  nineteen  of  them  stated  they  had  communication  barriers.  Reasons 
for  communication  barriers  for  these  individuals  included  personality  issues,  turnover, 
lack  of  understanding,  and  the  lack  of  availability  of  senior  leaders.  Reasons  are  located 
in  2(a),  2(b),  and  2(c). 
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Table  5  Communication  between  the  PM  and  Customer 


Themes 

#  of  responses 

%  of  participants 

#  of  participants 

(out  of  1 15) 

1.  Communication  Effectiveness 

23 

Positive 

102 

87% 

23 

Negative 

13 

13% 

23 

(out  of  23) 

2.  Communication  Issues 

19 

83% 

23 

(a)  MAJCOM/Program  Office  Turnover 

5 

22% 

23 

(b)  Understanding  among  stakeholders 

5 

22% 

23 

(c)  Personality 

5 

22% 

23 

Best  Practice  Three:  Identifying  the  Customer  Early 
(Q3- 1)  Do  you  know  who  your  customer  is? 


All  twenty-three  respondents  knew  who  their  customer  was  and  was  able  to 


identify  them.  The  majority  of  respondents  viewed  their  customer  as  the  MAJCOM, 


Program  Office,  DEPOT,  and  Requirements  community.  They  all  agreed  they  had 


multiple  customers  as  well. 


(Q3-2)  Is  your  effort  a  technology  push  or  technology  pull  effort? 


Nineteen  individuals  agreed  that  their  effort  was  technology  push  while  two 


viewed  the  effort  as  technology  pull.  Three  out  of  twenty-three  stated  their  technology 


was  identified  through  technology  pull.  Two  out  of  these  three  individuals  were  from  the 


same  directorate.  One  respondent  said  it  was  a  combination  of  both.  The  purpose  for  this 


question  was  to  understand  the  networking  ability  of  the  PM.  After  conducting  the 


literature  review  it  became  apparent  that  the  S&T  PM  needs  good  networking  abilities  to 


transition  technology. 


(Q3-3)  Was  the  customer  identified  early  on  your  project? 


Throughout  the  literature  review,  the  program  managers’  ability  to  identify  the 


customer  early  on  seemed  critical  in  the  technology  transition  process.  All  twenty-three 
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respondents  agreed  that  the  customer  was  identified  early  on  in  the  project.  They 
attributed  this  to  the  constant  communication  and  shared  understanding  with  the 
customer. 

Two  common  themes  from  this  section  emerged.  One  theme  was  that  PMs 
identified  their  customers  early  on  in  the  project  and  typically  pursued  them.  Results  for 
these  themes  can  be  found  in  Table  6.  The  underlining  meaning  for  this  theme  was  that 
the  S&T  PM  felt  if  they  did  not  market  their  technology  then  it  would  likely  not  be 
transitioned.  Ninety-five  percent  of  these  individuals  stated  PMs  of  6.3  projects  must  be 
able  to  network  themselves  to  other  organizations,  and  if  they  do  not  have  the  ability  to 
do  that  then  they  should  not  be  a  6.3  program  manager.  In  other  words,  S&T  PMs  of  6.3 
projects  should  be  a  people  person. 

The  second  theme  that  developed  from  this  section  was  that  PMs  were  reaching 
out  to  their  customer.  The  majority  of  the  PMs  believed  their  effort  was  technology  push. 
This  meant  the  PMs  were  collaborating  and  networking  with  their  customers. 

Table  6  The  PMs  Ability  to  Network 


Themes 

#  of  responses 

%  of  participants 

#  of  participants 

1.  Customer  identified 

23 

100% 

23 

Customer  identified  early 

23 

100% 

23 

2.  Technology  push 

19 

83% 

23 

(a)  Technology  pull 

3 

13% 

23 

(b)  Both 

1 

4% 

23 

Best  Practice  Four:  Participation 

(Q4-1)  Does  you  customer  actively  participate  on  a  regular  basis?  (meetings,  test 
events,  and  Program  Management  Reviews  (PMRs)) 

Five  out  of  twenty-three  said  the  customer  did  not  participate  enough.  Eighteen 
respondents  believed  the  customer  adequately  participated  on  a  regular  basis.  Even 
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though  the  majority  of  those  interviewed  stated  the  customer  participated  on  a  regular 
basis,  the  general  consensus  was  that  it  was  hard  to  get  all  the  stakeholders  to  participate 
on  a  regular  basis  due  to  busy  schedules. 

(Q4-2)  Do  necessary  acquisition  personnel  participate  when  needed  or  on  a 

regular  basis? 

In  the  literature  review  it  was  obvious  the  acquisition  community  was  a  key 
stakeholder  in  transitioning  technologies.  It  was  important  to  gain  an  understanding  of 
the  relationship  between  the  PM  and  one  of  the  key  stakeholders.  Eleven  out  of  twenty- 
three  respondents  thought  the  acquisition  community  did  not  participate  when  needed  or 
on  regular  basis.  Eleven  individuals  thought  the  acquisition  community  did  participate 
when  needed.  One  subject  stated  that  the  acquisition  community  participated  50%  of  the 
time. 

(Q4-3)  How  would  you  describe  your  relationship  with  the  acquisition 

community? 

The  majority  of  the  respondents  felt  they  had  a  decent  relationship  with  the 
acquisition  community.  Six  out  of  twenty-three  subjects  believed  their  relationship  with 
the  acquisition  community  was  not  good.  The  other  seventeen  respondents  felt  they  had 
a  good  relationship  with  the  acquisition  community. 

(Q4-4)  Could  the  relationship  be  better? 

Twenty-two  respondents  thought  the  relationship  could  be  better  with  the 
acquisition  community.  Three  of  the  participants  thought  it  could  be  better  but  did  not 
elaborate.  Five  respondents  believed  the  relationship  could  be  better  if  the  program 
offices  had  a  better  understanding  of  the  technology  capabilities  they  were  attempting  to 
transition.  Six  individuals  believed  more  communication  was  needed  between  the 


52 


laboratories  and  the  program  offices.  Two  of  the  respondents  felt  that  senior  leaders  from 
both  communities  needed  to  be  more  involved.  Another  two  respondents  believed  that 
organizational  barriers  and  stereotypes  needed  to  be  broken.  One  respondent  did  not 
answer  the  question.  Two  respondents  thought  there  needed  to  be  more  trust  from  both 
communities.  One  respondent  suggested  PMs  need  more  experience  in  the  program 
office  to  gain  a  better  understanding  on  the  acquisition  community.  One  individual 
thought  it  could  be  enhanced  if  the  S&T  PM  had  a  better  understanding  of  the  program 
office  requirements. 

(Q4-5)  What  type  of  interaction  do  you  have  with  industry? 

Again,  this  question  was  to  gain  an  understanding  of  the  networking  ability  the 
PM  has.  Twenty-one  respondents  stated  they  actively  interact  with  industry  on  regular 
basis.  Two  respondents  said  they  do  not,  but  agree  it  is  important. 

(Q4-6)  Do  you  have  any  documents  that  outline  key  stakeholders  roles  and 

responsibilities  and  how  they  should  be  participating  on  the  effort? 

The  purpose  of  this  question  was  to  gain  insight  into  the  collaboration  methods 
the  program  manager  used.  Four  respondents  said  they  had  no  documents  that  outlined 
responsibilities.  Eight  respondents  stated  they  had  some  type  of  an  informal  agreement. 
Eleven  of  those  interviewed  said  they  had  a  formal  structure  that  described  roles  and 
responsibilities. 

(Q4-7)  If  you  have  an  outstanding  relationship  with  your  customer  what  would 

you  attribute  to  that  strong  relationship? 

Ten  out  of  the  twenty-three  subjects  interviewed  though  that  communication  was 
the  number  one  reason  for  the  good  relationship.  Thirteen  of  the  twenty-three  attributed 
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understanding  among  the  key  stakeholders  as  the  number  one  reason  for  a  good 
relationship. 

Different  themes  emerged  from  this  section  of  questions  as  well.  Table  7 
illustrates  the  results  for  theme  one.  Of  the  PMs  interviewed,  48%  felt  the  acquisition 
community  did  not  participate  enough.  The  absent  participation  was  attributed  to  the 
busy  schedule  of  the  program  offices,  turnover,  and  a  lack  of  interest  in  the  technology. 
The  PMs  interviewed  stated  the  lack  of  interest  was  due  to  the  program  offices  only 
caring  about  what  the  technology  could  do  for  them  short  tenn  instead  of  long  tenn. 


Table  7  Program  Office  Participation 


Themes 

#  of  responses 

%  of  participants 

#  of  participants 

1 .  Acquisition  personnel  participated  when  needed 

11 

48% 

23 

Acquisition  personnel  did  not  participate  when  needed 

11 

48% 

23 

50%  of  the  time 

1 

4% 

23 

Another  theme  that  emerged  was  the  lack  of  formal  agreements  the  PMs  had  with 
their  stakeholders.  Table  8  illustrates  the  results  for  theme  two.  Out  of  the  twenty-three 
individuals  interviewed  52%  either  had  no  agreements  with  their  customers,  or  had  no 
formal  documentation  that  outlined  roles  and  responsibilities,  ft  is  also  important  to  note 
that  48%  of  those  individuals  who  stated  the  acquisition  community  did  not  participate 
enough  were  part  of  the  52%  of  those  who  had  no  formal  arrangements  with  their 
stakeholders. 


Table  8  Roles  and  Responsibilities 


Themes 

#  of  responses 

%  of  participants 

#  of  participants 

2.  Formal  arrangements  with  stakeholders 

11 

48% 

23 

No  formal  arrangements 

8 

35% 

23 

No  arrangements  at  all 

4 

17% 

23 
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Best  Practice  Five:  Knowledge  Management 

(Q5-1)  Would  you  say  that  the  majority  of  the  PMs  in  AFRL  are  familiar  with 
acquisition  processes?  (i.e.  Schedule,  POM,  PMRs,  requirements  generation, 
financial  management,  and  defense  acquisition) 

Twelve  out  of  the  twenty-three  individuals  interviewed  believed  program 

managers  within  AFRL  were  familiar  enough  with  acquisition  processes  to  do  their  job. 

The  other  1 1  individuals  interviewed  did  not  think  they  understood  the  acquisition 

processes  enough  to  do  their  job.  The  general  consensus  was  the  PMs  thought  they 

needed  more  training  and  program  office  experience.  Even  those  PMs  that  stated  they 

were  familiar  enough  with  the  acquisition  processes  thought  they  needed  refresher 

courses  with  program  management  and  acquisition  processes.  After  asking  this  question 

the  researcher  realized  that  S&T  PMs  did  not  need  to  be  Level  One  or  Level  Two 

certified  in  program  management.  There  is  a  certification  process,  but  program 

management  certification  is  not  requirement  for  S&T  program  managers. 

(Q5-2)  Do  you  know  early  on  or  even  at  all  if  your  project  can  be  transitioned  or 
inserted  into  a  legacy  system? 

This  question  was  developed  as  a  lead  in  to  the  following  question.  The  purpose 

was  to  get  the  PM  thinking  about  how  they  view  transition  windows  for  defense  systems 

that  are  managed  by  the  acquisition  community.  Ten  out  of  the  twenty-three  respondents 

typically  could  not  tell  if  their  technology  could  be  transitioned.  Thirteen  of  the 

individuals  believed  they  could  judge  whether  their  project  could  be  transitioned. 

(Q5-3)  Do  you  base  your  technology  availability  dates  on  weapon  system 
transition  windows? 

The  majority  of  those  interviewed  did  not  forecast  dates  their  technology  might  be 
able  to  transition.  Fourteen  individuals  did  not  base  their  project  schedule  on  weapon 
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system  transition  windows.  Eight  individuals  did  base  their  schedules  on  weapon  system 
transition  windows.  One  stated  they  did  this  50%  of  the  time. 

(Q5-4)  Do  you  understand  how  your  technology  fits  in  providing  new  warfighter 

capabilities? 

All  twenty-three  subjects  interviewed  understood  how  their  technology  provided 
the  warfighter  new  capabilities.  The  PMs  were  very  comfortable  and  confident  about  the 
technical  capabilities  their  technology  brought  to  the  warfighter. 

(Q5-5)  Can  you  very  clearly  define  the  “as-is”  environment  and  then  describe  the 

environment  with  your  technologies  implemented? 

All  twenty-three  agreed  they  understood  the  environment  now  with  the  legacy 
system  and  how  it  would  change  after  the  implementation  of  their  technology.  Again,  the 
participants  were  confident  about  how  their  technology  would  make  a  difference  for  the 
warfighter. 

Two  themes  emerged  here  as  well.  Table  9  illustrates  the  results.  Out  of  the 
twenty-three  individuals  interviewed  48%  did  not  believe  AFRL  program  managers  were 
familiar  enough  with  the  acquisition  processes.  It  is  critical  that  the  S&T  PM  be  familiar 
with  the  acquisition  processes  because  the  acquisition  community  plays  a  major  role  in 
the  transition  of  technologies.  More  than  half  of  those  individuals  that  thought  PMs  were 
not  familiar  enough  with  acquisition  processes  attributed  it  to  the  lack  of  training  or 
program  office  experience.  This  theme  could  also  be  linked  to  the  other  theme  that 
emerged:  the  PMs  understanding  of  the  weapon  system  transition  windows. 

Out  of  the  twenty-three  PMs  interviewed  61%  did  not  look  at  future  weapon 
system  transition  windows  too  see  if  their  technology  could  be  inserted  into  a  legacy 
system.  The  majority  of  the  individuals  stated  they  did  not  have  the  necessary  data  to 
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make  those  forecasts.  The  PMs  felt  they  needed  schedules  and  roadmaps  provided  by  the 
program  offices  to  forecast  efficiently. 

Table  9  Acquisition  Processes 


Themes 

#  of  responses 

%  of  participants 

#  of  participants 

1 .  PMs  are  familiar  with  acquisition  processes 

12 

52% 

23 

PMs  are  not  familiar  with  acquisition  processes 

11 

48% 

23 

2.  Do  not  base  technology  availability  on  transition  windows 

14 

61% 

23 

Base  technology  availability  on  transition  windows 

8 

35% 

23 

50/50 

1 

4% 

23 

Best  Practice  Six:  Incentives 

(Q6-1)  What  incentives  do  scientists  or  PMs  on  product  development  teams  have 

to  reach  out  to  the  customer? 

The  purpose  of  this  question  encouraged  the  participant  to  think  about:  1)  Do  they 
need  incentives.  2)  What  incentives  they  had  with  which  to  network.  Four  PMs  agreed 
there  are  no  incentives  to  reach  out  to  the  customer.  Nineteen  out  of  twenty-three  felt 
that  the  only  reason  to  reach  out  to  your  customer  was  because  of  personal  motivation. 
The  majority  of  the  PMs  believed  they  had  to  be  internally  driven  to  manage  6.3  projects. 

(Q6-2)  What  incentives  do  you  have  to  transition  your  project? 

Four  out  of  twenty-three  said  the  only  reason  to  transition  was  for  personal 
satisfaction.  Another  group  of  four  individuals  thought  monetary  compensation  was  an 
incentive  they  had  currently.  Three  out  of  twenty-five  agreed  that  receiving  a  good  rating 
was  an  incentive.  One  subject  thought  recognition  was  an  incentive.  Another  individual 
thought  just  seeing  the  technology  transition  was  an  incentive.  One  subject  could  only 
see  a  disincentive,  because  once  a  PM  transitions  a  technology  they  must  start  all  over 
with  a  new  program.  Nine  individuals  stated  there  were  no  incentives  to  transition  a 
technology. 
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(Q6-3)  Do  you  have  any  suggestions  on  possible  incentives? 

Eight  out  of  the  twenty-three  individuals  believed  allowing  the  S&T  PM  to  get 
experience  in  a  program  office  was  a  start  in  the  right  direction.  Seven  individuals  did 
not  have  any  suggestions.  Four  out  of  twenty-three  agreed  that  a  better  monetary 
reimbursement  and  promotion  plan  needed  to  be  in  place.  Two  individuals  felt  that  a 
specific  transition  process  put  in  action  would  be  an  incentive.  One  individual  offered  an 
incentive  that  would  promote  individuals,  but  allow  them  to  stay  in  their  expertise  instead 
of  being  promoted  to  management.  One  of  the  subjects  thought  patriotism  was  their  only 
incentive.  They  felt  important  and  self-worth  if  their  technology  was  adopted  by  a 
warfighting  system. 

One  theme  that  emerged  in  this  section  was  that  PMs  did  not  believe  they  had  any 
incentives  given  to  them  by  the  organization  to  transition  a  technology.  Table  10 
illustrates  the  results.  After  twenty-three  PMs  were  interviewed  65%  stated  there  were  no 
organizational  incentives  to  transition  their  technology.  Five  of  those  individuals  stated 
the  only  incentive  was  personal  satisfaction.  One  individual  stated  there  was  only  a 
disincentive  to  transition  projects  because  once  a  technology  is  transitioned  that 
individual  had  to  find  a  new  job. 


Table  10  Incentives 


Themes 

#  of  responses 

%  of  participants 

#  of  participants 

No  organizational  incentives 

14 

61% 

23 

Monetary 

3 

13% 

23 

Rating  or  OPR 

3 

13% 

23 

Recognition 

2 

9% 

23 

Only  disincentives 

1 

4% 

23 

Best  Practice  Seven:  Affordability  Metrics 
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(Q7-1)  Do  you  use  affordability  metrics  for  your  project  schedule  and 

performance  criteria? 

Fourteen  out  of  twenty-three  participants  stated  they  did  not  use  affordability 
metrics.  Nine  individuals  did  say  they  use  some  type  of  affordability  metrics.  However, 
five  of  those  individuals  that  used  affordability  metrics  came  from  the  same  directorate. 

It  seemed  these  practices  were  utilized  more  in  some  directorates  than  others.  Some 
stated  that  it  was  a  requirement  for  them,  however  the  majority  did  not. 

(Q7-2)  How  do  you  determine  your  exit  criteria? 

Eleven  out  of  twenty-three  interviewed  stated  they  detennined  the  exit  criteria  by 
meeting  a  certain  percentage  of  Key  Performance  Parameters  (KPPs).  Three  individuals 
determined  their  exit  criteria  by  demonstrating  their  technology  in  a  relevant 
environment.  After  demonstrating  in  a  relevant  environment  the  results  had  to  be  in  a 
certain  percentage  range  with  a  pass/fail  rating.  It  is  important  to  note  the  individuals  that 
did  use  KPPs  also  demonstrated  their  technologies  in  a  relevant  environment.  Four  out  of 
twenty-three  said  the  used  the  TRL  gauge  to  detennine  the  exit  criteria.  The  individuals 
that  used  the  TRL  as  the  determining  factor  said  their  technology  could  not  meet  the  exit 
criteria  if  it  did  not  reach  a  TRL  6.  Meeting  TRL  6  is  also  in  accordance  with  AFI  61- 
101.  Three  out  of  twenty-three  did  not  determine  exit  criteria.  Two  individuals  stated  it 
was  a  “best  guess”  in  detennining  the  exit  criteria. 

(Q7-3)  Did  you  perform  a  Return  on  Investment  (ROI)  or  create  a  business  case 

at  the  start  of  your  technical  efforts? 

Twenty  out  of  twenty-three  personnel  interviewed  said  they  did  not  perform  any 
type  of  ROI  or  business  case.  Three  individuals  did  say  they  used  ROIs  and  business 
cases.  However,  one  of  the  individuals  that  did  perfonn  ROIs  and  business  cases  stated 
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they  were  ineffective  because  too  many  assumptions  go  into  the  data  and  they  did  not 
have  the  appropriate  personnel  to  perform  these. 

(Q7-4)  Do  you  have  the  right  personnel  to  measure  and  perform  these  aspects  of 

the  project?  (i.e.  Affordability  Metrics,  ROI,  IPPD,  and  Business  Cases) 

Twelve  out  of  the  twenty-three  participants  said  they  did  not  have  the  right 
personnel  to  perfonn  this  part  of  the  job.  Eleven  individuals  said  they  had  the  necessary 
personnel  to  do  this  job.  Four  out  of  the  eleven  subjects  that  did  say  yes  came  from  the 
same  directorate.  Throughout  the  interview  process  it  was  apparent  that  some  PMs  had 
sufficient  support  and  resources  available  and  some  did  not. 

Two  themes  did  emerge  out  of  this  section.  Results  for  both  themes  can  be  found 
in  Table  1 1.  One  theme  that  emerged  in  this  section  was  the  nonuse  of  affordability 
metrics.  Out  of  twenty-three  individuals  interviewed  61%  stated  they  did  not  use  these 
metrics.  Throughout  the  literature  review  the  use  of  affordability  metrics  was  stressed 
multiple  times  (A  Guide  for  S&T  Program  Managers,  2001,  DOD  500  2-R,  2002,  and 
The  Manager’s  Guide  to  Technology  Transition  in  an  Evolutionary  Acquisition 
Environment,  2003).  It  is  also  important  to  note  that  when  interviewees  were  asked  about 
the  use  of  specific  affordability  tools  (ROIs  and  Business  Case),  87%  stated  they  did  not 
use  these  tools. 

Another  common  theme  that  arose  was  that  52%  of  those  interviewed  believed 
they  did  not  have  adequate  personnel  to  perform  affordability  metrics.  These  same 
individuals  were  also  part  of  the  87%  that  did  not  use  ROIs  and  Business  Cases  as  well  as 
part  of  the  61%  that  stated  they  did  not  use  affordability  metrics  at  all.  The  resources 
seemed  to  be  available  in  some  organizations  and  absent  in  others.  It  is  also  important  to 
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note  that  if  the  S&T  PM  is  going  to  implement  affordability  metrics  such  as  an  ROI  or 
business  case,  they  will  need  certain  data  from  the  program  offices  and  support  from 
finance  departments  to  do  this  correctly. 


Table  11  Affordability  Metrics 


Themes 

#  of  responses 

%  of  participants 

#  of  participants 

1 .  Did  not  use  affordability  metrics 

14 

61% 

23 

Did  use  affordability  metrics 

9 

39% 

23 

Did  not  perform  ROIs  or  Business  Cases 

20 

87% 

23 

2.  Do  not  have  the  necessary  personnel  to  perform 
affordability  metrics 

12 

52% 

23 

Have  the  necessary  personnel  available 

11 

48% 

23 

Best  Practice  Eight:  Technology  Transition  Team 

(Q8-1)  Do  you  think  a  technology  transition  team  would  help  market/transition 

AFRL  projects? 

These  next  five  questions  were  an  attempt  to  see  if  program  managers  in  the 
laboratories  felt  they  needed  additional  help  with  transitioning  technologies.  Three  out  of 
twenty-three  subjects  interviewed  said  they  did  not  think  this  team  could  be  effective. 
However,  twenty  individuals  interviewed  did  think  this  was  a  good  concept  and  that  it 
could  be  effective.  Throughout  these  questions  from  this  section  it  was  clear  the  program 
managers  felt  they  needed  help  in  transitioning  technologies. 

(Q8-2)  Do  you  think  PMs  in  the  labs  have  the  right  skills  to  do  PM  work? 

Fourteen  of  those  interviewed  believed  that  the  PMs  within  AFRL  do  not  have  the 
necessary  skills  to  do  the  program  management  side  of  the  job.  Half  of  those  fourteen 
attributed  it  to  the  lack  of  training  and  guidance.  Nine  out  of  twenty-three  participants 
did  believe  the  PMs  within  the  labs  have  the  necessary  skills  to  do  program  management 
duties. 
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(Q8-3)  Do  the  product  development  teams  have  the  right  expertise/personnel  to 
facilitate  technology  transition  ? 

Nine  out  of  twenty-three  said  they  did  not  have  the  right  personnel  needed  to 

effectively  transition  technology.  Seven  people  stated  the  resources  are  available,  they 

just  are  not  immediate.  Another  seven  thought  they  did  have  the  necessary  personnel 

needed  to  facilitate  technology  transition. 

(Q8-4)  Are  their  other  career  fields  that  could  be  of  benefit  to  help  projects 
transition? 

Four  out  of  twenty-three  did  not  think  any  other  career  fields  could  come  in  and 

be  of  benefit.  However,  two  of  those  four  individuals  thought  the  S&T  program  manager 

needed  more  training  in  acquisition.  Nineteen  of  the  participants  did  believe  other  career 

fields  could  come  in  and  be  of  benefit  and  had  a  variety  of  suggestions.  Nine  out  of  the 

twenty-three  individuals  thought  a  person  with  a  marketing  background  could  be  of 

benefit.  Seven  participants  thought  a  Subject  Matter  Expert  (SME)  would  help  out  a 

great  deal.  A  SME  is  a  person  with  experience  in  an  operational  setting  and  is  capable  of 

providing  an  end  user  perspective.  Two  individuals  thought  a  person  with  a  couple  of 

years  of  program  office  experience  would  be  a  good  addition.  One  subject  saw  a  need  for 

a  person  with  the  ability  to  conduct  ROIs  and  business  cases. 

(Q8-5)  Would  you  be  in  favor  of  having  someone  else  work  the  transition  issues 
on  your  program? 

Fourteen  of  the  twenty-three  interviewed  favored  someone  else  working  transition 
issues  for  their  project.  Nine  out  of  the  twenty-three  individuals  did  not  think  this  would 
be  a  good  approach.  All  nine  of  those  individuals  did  not  think  another  organization  or 
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personnel  set  could  be  effective  because  they  would  not  be  familiar  enough  with  the 
project. 

Multiple  themes  emerged  from  this  section.  Results  for  themes  one  and  two  can 
be  found  in  Table  12.  Of  the  twenty-three  individuals  interviewed,  87%  thought  the 
implementation  of  a  technology  transition  team  would  be  effective.  The  technology 
transition  team  was  an  idea  taken  from  the  National  research  Council  (2004).  One  of  the 
three  best  practices  the  NRC  suggested  was  the  creation  of  an  environment  that  fostered 
technology  transition.  Having  the  right  people,  skills,  and  abilities  is  one  step  to  attaining 
that  environment.  After  the  first  10  interviews  it  was  obvious  that  program  managers 
needed  help  from  other  personnel  with  specific  skill  sets  to  transition  technology.  This 
idea  also  applies  to  the  next  theme  which  deals  with  the  program  managers’  team  having 
the  necessary  personnel  to  facilitate  technology  transition. 

Out  of  twenty-three  individuals  interviewed,  70%  had  negative  responses 
associated  with  PMs  having  the  right  personnel  needed  to  facilitate  technology  transition. 
Nine  individuals  did  not  believe  they  had  the  necessary  personnel,  but  did  not  elaborate 
on  the  question.  Seven  of  those  interviewed  thought  the  resources  might  be  there,  but  not 
immediately.  Having  the  right  people  in  the  right  place  at  the  right  time  is  not  a  new 
concept.  The  importance  of  this  question  was  to  gain  insight  into  PMs  having  the 
necessary  support  when  working  transitional  issues. 


Table  12  Transition  Team 
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Themes 

#  of  responses 

%  of  participants 

#  of  participants 

1.  Believe  a  transition  team  would  be  of  benefit 

20 

87% 

23 

Did  not  think  a  transition  team  would  help 

3 

13% 

23 

2.  PMs  do  not  have  the  right  personnel 

9 

39% 

23 

Personnel  is  available  but  not  immediate 

7 

30% 

23 

PMs  do  have  the  right  personnel 

7 

30% 

23 

Another  theme  was  the  suggestion  that  other  career  fields  could  be  of  benefit  to 
the  PM.  Results  for  themes  three  and  four  can  be  found  in  Table  13.  After  twenty-three 
interviews,  83%  of  the  respondents  thought  other  career  fields,  if  added  to  the  IPT  or 
development  team,  could  be  a  real  benefit  when  working  transition  issues.  Of  the 
nineteen  individuals  that  responded  positively,  39%  believed  an  individual  with  a 
marketing  degree  would  help.  In  an  earlier  question,  83%  believed  their  effort  was 
technology  push.  The  importance  here  is  even  though  83%  pushed  their  technology  to 
the  customer,  almost  half  of  those  individuals  stated  they  needed  more  help  in  the 
technology  push  effort. 

The  last  theme  from  this  section  was  that  more  than  half  (61%)  felt  program 
managers  did  not  have  the  necessary  skills  to  perform  the  program  management  part  of 
the  job.  Half  of  these  individuals  attributed  this  to  the  lack  of  training  and  guidance. 
Throughout  these  questions  in  this  section  it  was  apparent  that  the  majority  of  the  PMs 
interviewed  saw  value  in  adding  other  necessary  skill  sets  to  the  team  as  well  as 
necessary  training  in  acquisition  processes. 


Table  13  Other  Skill  Sets  to  Aid  the  PM 
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Themes 

#  of  responses 

%  of  participants 

#  of  participants 

3.  Could  other  career  fields  benefit  the  PM 

Yes 

19 

83% 

23 

No  other  career  fields  could  be  of  benefit 

Suggestions  for  career  fields  needed 

4 

17% 

23 

Marketing  person 

9 

39% 

23 

Subject  Matter  Expert  (SME) 

7 

30% 

23 

4.  PMs  do  not  have  the  necessary  skills  to  do  PM  work 

14 

61% 

23 

Lack  of  Training/Guidance 

7 

30% 

23 

PMs  do  have  the  necessary  skills  to  do  PM  work 

9 

39% 

23 

Best  Practice  Nine:  Ensuring  Technology  Maturity 

(Q9-1)  How  do  you  ensure  technology  maturity  for  your  project? 

Nine  out  of  twenty-three  individuals  ensured  technology  maturity  through 


demonstrations.  If  the  technology  could  not  demonstrate  certain  capabilities  or  interface 


with  the  necessary  components,  then  the  PM  did  consider  their  technology  mature 


enough.  Eight  individuals  measured  their  maturity  by  using  the  TRL  as  a  gauge.  One 


individual  did  nothing  to  ensure  maturity.  One  individual  stated  that  the  type  of  funding 


determined  the  technologies  maturity  level.  Two  out  of  twenty-three  participants  stated 


that  it  was  their  personal  assessment  that  ensured  technology  maturity.  One  subject  said 


it  was  through  sufficient  testing  that  ensured  technology  maturity.  One  manager  stated 


he/she  could  not  do  enough  analysis  to  gauge  the  maturity  of  the  technology. 


(Q9-2)  Is  the  Technology  Readiness  Level  (TRL)  an  appropriate  gauge  to 
measure  technology  maturity? 


The  majority  of  the  participants  did  think  the  TRL  was  a  good  tool  and  also  used 


the  tool  to  measure  technology  maturity.  Sixteen  out  of  twenty-three  participants 


believed  the  TRL  was  an  appropriate  tool  to  measure  technology  maturity.  Seven 


individuals  did  not  think  the  TRL  was  a  good  tool  to  measure  maturity.  These  seven 
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individuals  thought  the  TRL  tool  was  inconsistent  and  the  maturity  levels  could  easily  be 
inflated  to  better  market  the  technology. 

(Q9-3)  Do  you  track  and  assess  technology  maturation  by  key  milestones  or  just 

prepare  a  chart  for  Program  Baseline  Reviews  ( PBRs )? 

Six  out  of  twenty-three  did  not  use  key  milestones  or  PBRs.  However,  seventeen 
did  use  key  milestones  and  PBRs  to  track  the  maturation  of  their  technology.  It  was 
apparent  that  the  PMs  that  did  use  PBRs  and  key  milestones  communicated  with  their 
customer  on  a  regular  basis. 

(Q9-4)  Do  the  TRLs  help  you  in  communicating  with  your  customer  and  in 

meeting  their  expectations? 

Ten  out  of  the  twenty-three  participants  did  believe  TRLs  helped  in 
communicating  to  their  customer  what  they  were  delivering  and  how  mature  the 
technology  was.  Twelve  individuals  did  not  think  the  TRLs  helped  the  customer 
understand  what  they  were  getting.  One  individual  stated  it  was  a  50/50  chance  the 
customer  understood  the  TRLs.  The  individuals  that  did  not  think  TRLs  helped  in 
communicating  with  their  customer  also  maintained  that  the  customer  did  not  care  about 
the  maturity  levels.  The  interviews  concluded  the  PM  believed  the  customer  was  more 
concerned  with  when  they  were  getting  the  technology  than  anything  else. 

One  common  theme  that  emerged  from  this  section  of  questions  was  that  91%  of 
those  interviewed  valued  ensuring  the  maturity  of  the  technology.  Results  are  located  in 
Table  14.  As  it  was  established  in  Chapter  Two,  the  maturity  of  the  technology  is  very 
important.  If  immature  technologies  reach  the  demonstration  phase,  they  will  likely  fail. 
If  this  failure  occurs  the  technology  will  fall  prey  to  the  valley  of  death.  It  is  important  to 
note  that  91%  of  the  participants  did  use  the  TRLs  as  a  guide,  but  were  not  necessarily 
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primary  method  chosen.  Out  of  the  twenty-one  individuals  that  did  place  an  emphasis  on 


maturity,  43%  detennined  the  maturity  of  their  technology  through  demonstrations 


satisfying  certain  capabilities.  Of  those  individuals  who  ensured  technology  maturity, 


38%  used  the  TRL  definitions  as  the  main  gauge  for  maturity. 

Table  14  Ensuring  Technology  Maturity 


Themes 

#  of  responses 

%  of  participants 

#  of  participants 

Individuals  who  placed  an  emphasis  on  maturity 

21 

91% 

23 

PMs  ensured  maturity  through 

Maturity  through  demonstrations 

9 

39% 

23 

Maturity  based  on  TRL  definitions 

8 

35% 

23 

General  Overarching  Questions 

(Q 10-1)  In  your  opinion  what  do  you  think  the  single  most  important  roadblock  in 
transitioning  AFRL  projects? 

Eight  out  of  the  twenty-three  interviewed  thought  the  availability  of  funding  was 
the  biggest  roadblock.  It  is  important  to  note  that  all  individuals  interviewed  thought 
funding  was  a  serious  issue,  but  offered  other  reasons  for  transition  roadblocks.  Seven 
individuals  thought  the  single  most  important  roadblock  was  the  lack  of  understanding 
amongst  the  key  stakeholders.  Understanding  issues  to  the  program  managers  meant  the 
lack  of  effective  communication,  involvement  of  the  necessary  people,  and  personality 
issues.  One  individual  stated  the  roadblock  to  transition  was  the  lack  of  a  defined 
transition  process.  Another  subject  stated  the  process  to  get  transition  documents  signed 
off  on  was  too  long.  One  participant  stated  the  roadblock  was  the  inability  for  the  PM  to 
understand  how  their  technology  fit  into  transition  windows  for  the  program  office.  Two 
individuals  thought  the  transition  roadblock  was  the  inability  to  do  multiple  informal 
tests.  One  subject  believed  the  transition  roadblock  was  getting  buy  in  from  senior 
leaders. 
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(Q10-2)  Do  you  think  having  AFRL  representation  in  the  program  offices  could 
help  transition  AFRL  projects? 

Eleven  out  of  the  twenty-three  individuals  thought  AFRL  representation  in  the 

program  offices  would  help  a  great  deal.  The  majority  that  gave  positive  affirmation 

thought  this  would  give  the  program  offices  a  better  understanding  of  AFRL 

technologies,  while  bringing  the  two  communities  closer  together.  Twelve  of  the 

individuals  did  not  believe  AFRL  representation  in  the  program  offices  would  help. 

(Q 1 0-3)  Do  you  think  there  is  adequate  guidance  given  to  you  in  order  to 
facilitate  technology  transition  ? 

Two  individuals  did  not  have  an  answer  for  this  question.  However,  thirteen  of 

those  interviewed  did  not  think  there  was  adequate  guidance  for  the  S&T  program 

manager.  Eight  out  of  the  twenty-three  participants  did  believe  there  was  adequate 

guidance  to  facilitate  technology  transition. 

(Q10-4)  What  do  you  think  the  process  is  that  sells  your  project;  i.e.  transitions  it 
or  markets  it? 

Six  out  of  twenty-three  individuals  stated  it  was  the  constant  networking  with  key 
stakeholders  that  marketed  their  technology.  Three  individuals  saw  senior  leaders  from 
different  organizations  as  the  avenue  that  transitioned  their  technology.  Four  out  of  the 
twenty-three  participants  thought  it  was  the  PM  marketing  the  technology  to  the  Action 
Officers  (AOs)  within  the  MAJCOMs.  Another  four  participants  thought  the  PM  must 
sell  the  idea  to  the  prime  contractor  first,  or  the  other  key  stakeholders  (MAJCOMS  and 
Program  Offices)  would  not  buy  in  to  the  technology.  One  subject  stated  the  process  was 
demonstrating  a  technology  with  a  focus  on  interoperability  with  other  systems.  One 
individual  stated  that  it  was  the  PMs  knowledge  on  how  to  transition  technology. 
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Another  individual  said  it  was  the  early  involvement  with  the  program  office  that 
transitioned  the  technology.  One  subject  thought  it  was  getting  buy  in  from  the  lower 
level  worker  so  the  subordinates  could  advocate  for  that  technology  to  upper 
management.  One  PM  stated  it  was  waiting  for  the  MAJCOMs  to  come  to  them 
(technology  pull).  The  last  individual  stated  it  was  constant  communication,  establishing 
a  relationship,  and  building  trust  with  the  customer  transitioned  the  technology. 

(Q10-5)  Do  you  think  the  ATC  does  a  good  job  to  ensure  technologies  are 

transitioned? 

Nine  of  those  interviewed  believed  the  ATC  did  a  good  job  in  facilitating 
technology  transition.  Five  out  of  the  twenty-three  participants  did  not  have  an  answer 
for  this  question  due  to  inexperience  with  the  ATC.  Nine  of  the  participants  did  not  think 
the  ATC  ensured  technologies  were  transitioned.  Those  nine  individuals  felt  that  the 
ATC  allowed  their  technology  to  get  exposure  to  senior  leaders  and  other  organizations, 
but  did  not  ensure  the  technology  could  get  transitioned. 

(Q 1 0-6)  Do  you  think  the  Applied  Technology  Council  process  works  to  prevent 

delays  in  transition? 

Eleven  of  those  interviewed  did  not  think  the  ATC  prevented  delays  in  transition. 
Eight  out  of  the  twenty-three  subjects  interviewed  did  not  have  an  answer  for  this 
question.  Four  individuals  believed  the  ATC  does  prevent  delays  in  transition. 

(Q 1 0-7)  How  do  you  go  about  getting  a  champion  for  your  project? 

Eight  out  of  the  twenty-three  participants  thought  getting  buy  in  from  the 
MAJCOMs  first  got  them  a  champion  for  their  technology.  Two  individuals  believed 
soliciting  program  offices  to  advocate  for  their  technology  would  typically  champion  the 
project.  Six  out  of  the  twenty-three  individuals  thought  getting  the  right  people  involved 
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from  the  necessary  organizations  (MAJCOMs,  Program  Offices,  and  Contractor  etc.), 
including  decision  makers,  got  a  champion  for  their  technology.  Two  of  those 
interviewed  stated  they  typically  marketed  their  project  to  subordinates  of  decision 
makers  so  the  lower  level  worker  could  advocate  for  the  technology.  Another  two 
participants  thought  they  had  a  champion  when  leadership  from  different  organizations 
discussed  the  technology.  Two  out  of  the  twenty-three  participants  said  they  procured 
people  willing  to  champion  their  project  because  of  their  technology  merit.  One 
individual  stated  it  was  the  ability  to  obtain  the  necessary  data  to  conduct  a  reliable  ROI 
that  inspired  personnel  from  program  offices  and  MAJCOMs  to  champion  the 
technology. 

(Q 1 0-8)  Which  organization  is  the  most  difficult  to  get  buy  in  from? 

Fifteen  out  of  the  twenty-three  program  managers  interviewed  thought  the 
program  office  posed  the  most  difficulty  in  getting  buy  in.  One  individual  stated  the 
prime  contractor  was  the  hardest  from  whom  to  get  a  buy  in,  while  four  participants 
agreed  that  the  MAJCOM  was  most  difficult.  Three  out  of  the  twenty-three  individuals 
stated  AFRL  Headquarters  was  the  most  difficult  to  get  buy  in  from. 

This  chapter  has  synthesized  the  data  collected  into  themes  by  interpreting  views 
of  program  managers  within  AFRL.  After  analyzing  the  result  the  data  revealed  that  the 
S&T  program  managers  did  have  good  relationships  with  their  customers,  but  the  PMs 
still  need  adequate  training,  guidance,  and  a  dynamic  team  in  order  to  transition 
technology  efficiently.  After  the  data  was  analyzed  and  broken  into  themes,  the 
researcher  observed  that  that  some  best  practices  were  being  used  and  some  were  not. 
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Synthesis  of  themes  that  emerged  during  the  data  analysis  portion  should  provide  a  solid 
foundation  to  draw  conclusions  and  recommendations  for  transitioning  technology. 
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V.  CONCLUSIONS  AND  RECOMMENDATIONS 


5.1  INTRODUCTION 

The  purpose  of  this  research  was  to  determine  what  extent  AFRL  was 
implementing  known  technology  transition  best  practices.  Technology  transition  best 
practices  were  taken  from  Industry  and  DOD  sponsored  publications.  Program  managers 
of  6.3  projects  were  interviewed  to  gain  insight  into  technology  transition  guidance  being 
practiced.  This  research  was  also  an  attempt  to  gain  an  understanding  into  how  effective 
these  best  practices  were  for  the  program  managers  using  them.  This  chapter  will 
provide  conclusions  and  recommendations  based  on  the  insight  given  from  program 
managers  within  AFRL.  Also,  this  chapter  will  provide  recommendations  for  future 
research. 

5.2  CONCLUSIONS 

Throughout  the  literature  review  the  researcher  was  able  to  answer  investigative 
question  one,  which  is: 

What  guidance  exists  for  technology  transition  within  the  DOD? 

There  were  multiple  documents  that  listed  key  elements  and  guidelines  for  transitioning 
technologies.  While  guidance  on  technology  transition  does  exist,  many  of  the 
documents  are  not  directive;  they  are  just  suggestive  in  nature.  Many  best  practices  such 
as  IT,  transition  team,  communication,  and  participation  overlapped  within  the 
documents.  Some  documents  were  tailored  to  specific  technologies  such  as  materials, 
while  another  focused  on  affordability.  All  of  the  documents  that  were  collected  from  the 
DOD  and  Industry  contributed  to  guidance  on  technology  transition.  Each  document 
played  an  important  role  in  the  development  of  the  best  practices. 
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After  conducting  the  interviews,  the  researcher  determined  a  deficiency  in  the 
implementation  of  guidance  and  training  for  S&T  program  managers.  The  conclusions 
drawn  here  answer  investigative  question  three: 

Do  the  managers  in  the  S&T  community  believe  the  current  guidance  is  adequate 

to  effectively  transition  technologies  to  the  warfighter? 

The  majority  of  those  interviewed  (62%)  did  not  believe  there  was  adequate  guidance  on 
how  to  transition  technologies.  After  talking  with  all  the  program  managers,  the 
researcher  discerned  PMs  learned  more  about  technology  transition  just  by  doing. 

Talking  with  many  of  the  program  managers,  I  realized  there  was  no  requirement  for  the 
PM  to  be  Level  One  or  Level  Two  certified  in  program  management.  As  it  was  stated  in 
Chapter  Four,  there  is  a  certification  process  for  program  management,  but  it  is  not  a 
current  requirement  for  PMs  in  the  S&T  community.  Training  and  certification  is  left  up 
to  the  individual  to  decide  whether  they  want  to  take  courses  dealing  with  program 
management.  Table  15  illustrates  the  importance  for  effective  guidance  and  training. 
Individuals  that  did  have  program  office  experience  claimed  without  that  experience  they 
would  be  lost  as  a  PM  in  AFRL.  To  clarify,  I  am  not  saying  that  AFRL  program 
managers  need  to  be  just  like  the  program  managers  in  the  acquisition  community,  but 
they  do  need  to  be  familiar  with  how  program  offices  operate  because  the  acquisition 
community  is  a  critical  stakeholder  in  transitioning  technology.  S&T  PMs  need  to 
understand  how  to  look  for  transition  windows  for  weapon  systems.  The  majority  of  the 
program  managers  interviewed  did  not  forecast  for  transition  windows.  Program  offices 
which  take  a  spiral  approach  to  development  presents  a  good  opportunity  for  PMs  in  the 
laboratories  to  see  how  their  technology  can  fit  in  that  window  of  spiral  development. 
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The  lack  of  involvement  from  other  organizations  could  be  due  to  the  perceived  lack  of 


guidance.  Correct  guidance  can  assist  S&T  program  managers  in  how  to  involve  other 


organizations.  This  is  critical  because  there  are  many  other  organizations  involved  in  the 


technology  transition  process. 

Table  15  Transition  Guidance/Training 


Themes 

#  of  responses 

%  of  participants 

#  of  participants 

There  is  not  adequate  guidance  on  technology  transition 

13 

62% 

21 

Acquisition  personnel  did  not  participate  when  needed 
Nothing  formal  that  defines  roles/responsibilities  of  the 

11 

48% 

23 

stakeholders 

12 

52% 

23 

PMs  are  not  familiar  with  acquisition  processes 

11 

48% 

23 

Need  more  PM  training/program  office  experience 

6 

26% 

23 

Do  not  base  technology  availability  on  transition  windows 

14 

61% 

23 

Did  not  use  affordability  metrics 

14 

61% 

23 

PMs  do  not  have  the  necessary  skills  to  do  PM  work 

14 

61% 

23 

Lack  of  Training/Guidance 

7 

30% 

23 

The  second  conclusion  that  could  be  drawn  from  this  research  was  the  need  for 


necessary  skill  sets.  After  talking  to  the  PMs,  the  researcher  ascertained  they  needed 
more  personnel  available  to  help  them  than  what  was  immediately  available  to  them. 
Table  16  illustrates  the  probable  needs  of  the  S&T  PM.  Program  managers  were  very 
quick  to  offer  suggestions  of  other  career  fields  that  could  be  of  benefit  to  them.  Many  of 
them  also  believed  if  they  had  the  necessary  personnel  with  specific  skill  sets  they  would 
be  able  to  efficiently  perform  affordability  metrics.  They  did  not  just  need  help  with 
affordability  metrics,  but  help  with  marketing  their  technology  as  well.  In  the  process  of 
my  analysis  it  became  obvious  that  if  PMs  were  not  somewhat  extra  verted,  then  they  had 
a  hard  time  marketing  their  technology.  This  in  turn,  would  pose  difficulty  for  them 
getting  buy  in  from  other  organizations.  Many  PMs  affirmed  having  some  type  of  AFRL 
representation  in  the  program  offices  would  help  facilitate  more  of  an  understanding 
between  both  communities.  Some  of  the  PMs  had  experience  in  their  last  job  working 
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with  an  AFRL  representative  that  was  co-located  in  a  program  office.  PMs  that  had  this 
experience  stated  it  helped  out  a  great  deal.  This  suggestion  was  also  consistent  with  one 
of  the  general  questions  which  asked,  “What  organization  poses  the  most  difficulty  in 
getting  buy  in  from.”  Out  of  the  twenty  three  PMs  interviewed,  65%  felt  the  program 


office  was  the  most  difficult. 

Table  16  S&T  Program  Managers  Needs 


Themes 

#  of  responses 

%  of  participants 

#  of  participants 

Do  not  have  the  necessary  personnel  to  perform  affordability 
metrics 

12 

52% 

23 

Believe  a  transition  team  would  be  of  benefit 

20 

87% 

23 

PMs  do  not  have  the  right  personnel 

9 

39% 

23 

Personnel  is  available  but  not  immediate 

7 

30% 

23 

Could  other  career  fields  benefit  the  PM 

Yes  other  career  fields  could  be  of  benefit 

19 

83% 

23 

Marketing  person 

9 

39% 

23 

Subject  Matter  Expert  (SME) 

7 

30% 

23 

AFRL  representation  in  program  offices 

11 

48% 

23 

The  third  conclusion  that  could  be  drawn  from  this  research  was  the  lack  of  a 


defined  process.  This  conclusion  answers  investigative  question  two: 

What  is/are  the  process  or  processes  that  exist(s)  to  transition  technologies  to  the 

warfighter? 

As  it  was  stated  earlier,  62%  believed  there  was  not  adequate  guidance  on  technology 
transition.  When  asked,  there  were  ten  different  responses  (43%)  to  what  PMs  thought 
the  process  was  that  transitioned  their  technology.  Table  17  illustrates  a  need  for  a 
defined  process.  Throughout  the  interviews,  the  PMs  implied  that  the  only  way  they 
knew  what  to  do  was  by  doing,  or  making  phone  calls  to  people  who  had  experience  and 
the  knowledge  to  help  them.  Many  of  them  even  stated  there  was  no  document  they 
could  pick  up  and  learn  what  the  technology  transition  process  is.  However,  it  did  seem 
that  different  directorates  had  different  types  of  customers,  which  might  call  for  different 
processes.  There  could  be  a  reason  for  not  having  a  defined  process  on  technology 
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transition.  After  talking  with  the  PMs  in  AFRL,  the  researcher  concludes  that  all  of  the 
actions  in  Table  17  need  to  happen  for  technologies  to  be  transitioned  from  the  S&T 
community  to  the  acquisition  workforce. 

Table  17  No  Defined  Process 


Themes 

#  of  responses 

%  of  participants 

#  of  participants 

There  is  not  adequate  guidance  on  technology  transition 

13 

62% 

21 

Networking  with  key  stakeholders 

6 

26% 

23 

Senior  leaders  talking  to  one  another 

3 

13% 

23 

Get  buy  in  from  MAJCOM  Action  Officers  (AO) 

4 

17% 

23 

Get  buy  in  from  prime  contractor  first 

4 

17% 

23 

Demonstrate  interoperability  with  other  systems 

1 

4% 

23 

PM  just  knowing  how  to  transition  the  technology 

1 

4% 

23 

Getting  program  office  involved  early 

1 

4% 

23 

Get  subordinates  on  customers  side  to  advocate  for  you 

1 

4% 

23 

Technology  pull 

1 

4% 

23 

Communicate  First-Establish  a  Relationship-Build  Trust 

1 

4% 

23 

The  fourth  conclusion  that  can  be  drawn  from  this  research  is  that  the  incentives 


for  transitioning  technologies  are  not  totally  efficient.  When  PMs  were  first  asked  about 
incentives  for  transitioning  technologies,  they  were  hard  pressed  to  come  up  with  an 
answer.  When  the  interviewees  thought  about  it,  65%  of  them  (Table  9)  did  not  know  of 
any  organizational  incentives.  After  talking  with  the  PMs,  the  researcher  discovered 
there  are  monetary  incentives  for  managers  who  transition  technologies,  however,  the 
PMs  who  acknowledged  there  was  a  monetary  incentive  revealed  that  the  dollar  amount 
was  very  small.  Through  the  interviews,  it  was  pointed  out  by  the  participants  that  PMs 
do  get  some  type  of  recognition  for  transitioning  technologies  as  well.  The  problem  with 
the  existing  incentive  program  is  that  it  is  not  currently  recognized  by  many  program 
managers  within  AFRL. 

The  fifth  conclusion  that  was  drawn  from  this  research  maintains  that  there  are 


technology  transition  best  practices  not  currently  being  implemented  that  could  be  of 


benefit  to  the  S&T  PM.  This  conclusion  addresses  investigative  question  four: 
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Is  there  technology  transition  practices  not  being  currently  implemented  in  AFRL 
that  could  enhance  the  transition  of  laboratory  technologies? 

After  talking  with  the  S&T  PMs  it  was  discovered  half  of  the  participants  (52%)  did  not 

have  formal  agreements  with  key  stakeholders,  which  outlined  roles  and  responsibilities. 

Throughout  the  literature  review,  it  was  apparent  that  having  some  type  of  formal 

agreements  with  the  key  stakeholders  would  facilitate  collaboration  among  the 

participants  (Dobbins,  2004,  National  Research  Council,  2004,  A  Guide  for  S&T 

Program  Managers,  2001,  AFI  61-101,  and  The  Manager’s  Guide  to  Technology 

Transition  in  an  Evolutionary  Acquisition  Environment,  2003).  The  majority  of  the 

program  managers  thought  it  was  critical  for  all  the  key  stakeholders  to  have  a  common 

understanding.  Some  type  of  formal  arrangements  that  all  the  key  stakeholders  agree  on 

would  help  facilitate  that.  It  was  established  earlier  that  the  incentives  currently  in  place 

are  not  that  effective.  Out  of  the  twenty  three  individuals  interviewed,  65%  did  not  know 

of  any  organizational  incentives  to  transition  technology.  Another  best  practice  that  was 

not  employed  in  full  was  the  implementation  of  a  technology  transition  team.  Many  of 

the  program  managers  thought  resources  were  limited  or  not  immediate.  Out  of  the 

twenty-three  program  managers,  52%  said  they  did  not  have  the  necessary  personnel  to 

perform  affordability  metrics,  while  87%  stated  they  did  not  even  perform  Return  on 

Investments  (ROIs)  or  business  cases.  They  simply  did  not  have  the  adequate  personnel 

to  do  ROIs  or  business  cases.  Most  of  the  individuals  interviewed  welcomed  the  idea  of 

having  other  personnel  with  specific  skill  sets  to  help  them  with  transition  issues. 
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5.3  RECOMMENDATIONS 


Based  on  the  insight  given  from  the  program  managers  that  were  interviewed,  the 
researcher  offers  a  few  recommendations.  The  first  recommendation  is  that  AFRL 
should  consider  ways  to  foster  an  environment  to  facilitate  technology  transition  through 
the  implementation  of  a  technology  transition  team.  This  team  should  include  members 
with  specific  skill  sets.  The  idea  of  the  transition  team  is  not  to  take  over  the  transition 
issues  for  the  PM,  but  to  help  them  in  the  transition  process.  The  transition  team  should 
include  members  that  have  program  office  experience.  The  general  consensuses  from  the 
interviews  were  PMs  had  difficulty  collaborating  with  program  offices  as  well  as  getting 
buy  in  from  them.  An  individual  with  program  office  experience  could  help  the  PM  deal 
with  these  issues.  Another  individual  that  should  be  included  is  one  with  a  financial 
background.  The  data  disclosed  that  program  managers  could  not  perfonn  certain 
affordability  metrics  that  could  be  of  benefit  to  them.  A  financial  person  could  ensure 
ROIs  or  business  cases  are  completed  for  the  6.3  projects.  Another  team  member  with  a 
marketing  background  would  offer  additional  skill  set.  From  the  data  gathered,  it  was 
obvious  the  PM  must  undertake  extensive  networking  with  multiple  organizations  when 
trying  to  transition  their  technologies.  A  marketing  person  would  be  of  considerable 
value  to  the  PM  and  their  efforts  to  reach  out  to  the  MAJCOMs,  program  offices,  and 
defense  contractors.  The  last  skill  set  that  might  be  of  great  benefit  to  the  team  would  be 
a  Subject  Matter  Expert  (SME).  Individuals  interviewed  repeatedly  suggested  the 
importance  of  a  SME.  A  SME  could  greatly  assist  the  PM  when  dealing  with  the  end 
user,  contractor,  MAJCOMs,  and  requirements  communities,  while  a  SME  with 
operational  experience  could  be  valuable  to  6.3  program  managers.  These  teams  should 
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be  stood  up  in  each  directorate  to  help  PMs  of  6.3  projects.  It  is  understood  that  current 
organizational  structures  and  hierarchy  might  not  be  able  to  accommodate  a  team  such  as 
this.  This  is  just  a  suggestion  from  the  researcher’s  point  of  view. 

Another  recommendation  is  for  the  directorates  to  consider  implementing  more 
co-locating  programs.  A  co-locate  program  is  where  an  S&T  PM  resides  in  a  program 
office  representing  a  certain  AFRL  directorate.  Some  of  the  PMs  interviewed  had 
experience  in  working  with  individuals  who  were  co-located.  Interviews  pointed  out  that 
co-located  individuals  facilitated  collaboration  between  the  laboratory  and  acquisition 
community.  The  majority  of  the  PMs  interviewed  (96%)  thought  their  relationship  with 
the  acquisition  community  could  be  better.  All  program  managers  believed  having  a 
general  understanding  amongst  the  key  stakeholders  was  essential  to  transition 
technology.  Out  of  the  twenty-three  individuals  interviewed,  48%  thought  having  some 
type  of  AFRL  representative  in  the  program  offices  would  help  the  PM  work  transition 
issues  with  the  acquisition  community.  More  directorates  that  take  advantage  of  the  co¬ 
locate  program  might  help  bridge  the  valley  of  death. 

A  third  recommendation  is  for  AFRL  to  consider  developing  a  data  storage 
warehouse  that  includes  technology  transition  lessons  learned,  techniques,  collaboration 
tools,  best  practices,  success  stories,  and  acquisition  processes  and  terminologies. 
Through  the  development  of  the  data  warehouse  AFRL  should  consider  publishing  a 
technology  transition  handbook.  The  handbook  should  serve  as  a  quick  reference  guide 
on  technology  transition.  There  should  be  a  brief  overview  on  programs  that  facilitate 
technology  transition.  Programs  should  include  SBIR,  MANTECH,  TTI,  ACTD, 
DARPA,  and  other  relevant  technology  transition  programs.  This  would  allow  the  6.3 
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program  manager  to  gain  insight  into  other  avenues  to  transition  technology.  The 
handbook  should  also  outline  specific  steps  the  PM  can  take  to  network  with  key 
stakeholders  such  as  the  acquisition  community,  end  user,  defense  contractor, 

MAJCOMs,  industry,  requirements  community,  and  financial  communities.  Available 
training  for  the  PM  should  also  be  listed  with  detailed  descriptions  of  the  training  and 
how  the  training  can  help  them  in  transitioning  technologies.  An  all  encompassing 
document  should  be  able  to  assist  the  program  manager  in  technology  transition. 

Another  recommendation  is  for  AFRL  to  readdress  their  incentive  program  for 
transitioning  technologies.  Many  of  the  program  managers  (65%)  were  not  aware  of  any 
organizational  incentives  to  transition  technology.  The  majority  of  the  program  managers 
were  personally  driven  to  transition  their  technology.  It  is  possible  that  monetary  rewards 
or  ratings  are  unimportant  to  S&T  program  managers.  AFRL  should  consider  incentives 
for  internally  driven  employees.  One  example  of  an  incentive  could  be  that  PMs  are  able 
to  transition  with  their  technology.  This  could  provide  a  valuable  resource  of  continuity 
to  a  technology’s  life-cycle. 

The  last  recommendation  pertains  to  training.  Sixty-one  percent  concurred  that 
program  managers  did  not  have  the  necessary  knowledge  to  do  the  PM  work.  Currently 
there  is  no  official  requirement  for  S&T  PMs  to  be  Level  One  or  Level  Two  certified  in 
program  management.  The  recommendation  is  that  S&T  PMs  be  at  least  Level  One 
certified  as  program  managers.  It  is  understood  that  the  certification  process  is  not 
attainable  overnight.  The  program  manager  must  take  the  required  PM  courses  as  well  as 
hold  a  position  of  program  manager  for  a  certain  amount  of  time.  This  certification  can 
only  help  the  S&T  program  manager  when  they  are  working  with  the  program  offices  and 
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acquisition  issues.  In  addition  to  acquisition  training  there  also  needs  to  be  refresher 
courses  once  a  year.  Acquisition  process  and  tenninology  are  constantly  changing. 

These  refresher  courses  should  minimize  problems  with  the  ongoing  changes  in  the 
acquisition  community  and  tenninology.  Procuring  S&T  program  managers  experience 
in  the  program  offices  could  be  another  fonn  of  training.  The  co-locate  program  is  just 
one  way  to  accomplish  this. 

5.4  LIMITATIONS 

There  were  limitations  that  arose  throughout  this  case  study.  The  first  limitation 
was  the  inability  for  the  researcher  to  obtain  interview  subjects  from  each  AFRL 
directorate.  Some  organizations  were  located  in  different  parts  of  the  country  which 
posed  difficulty  for  the  researcher  to  collectively  gain  insight  into  all  the  AFRL 
directorates.  The  second  limitation  was  the  availability  of  interview  subjects.  Multiple 
interviews  were  postponed  or  cancelled,  and  because  of  lack  of  participation  from 
scheduled  interviewees,  only  twenty-three  PMs  out  of  a  planned  thirty  PMs  were 
successfully  interviewed. 

5.5  RECOMMENDATIONS  FOR  FUTURE  RESEARCH 

Future  research  should  attempt  to  gather  additional  data  that  will  provide  further 
insight  from  other  audiences  such  as  the  program  offices,  MAJCOMs,  contractors,  Office 
of  the  Secretary  of  Defense  (OSD),  and  end  user  communities.  Detailed  and  extensive 
research  will  allow  for  a  more  in-depth  look  into  technology  transition. 

Based  on  the  data  and  analysis  from  this  research,  the  researcher  offers 
recommendations  for  future  study.  The  acquisition  community  was  established  to  be  a 
key  stakeholder  in  transitioning  laboratory  developed  technologies.  Research  in  the 
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future  should  attempt  to  gain  insight  on  working  relationships  and  processes  from  the 
acquisition  community’s  perspective.  There  were  obvious  difficulties  from  the  S&T  PMs 
perspective  in  dealing  with  the  program  offices.  For  future  research,  interview  subjects 
should  include  acquisition  personnel  that  have  had  experience  in  working  with  laboratory 
transitioned  technologies.  Acquisition  personnel  feedback  combined  with  this  research 
should  give  more  insight  into  how  to  bridge  the  gap  over  the  valley  of  death.  Other  areas 
for  future  research  include:  interview  subjects  that  represent  all  directorates  in  AFRL; 
explore  the  views  of  the  defense  contractor;  research  the  implementation  of  other 
technology  transition  programs  (MANTECH,  SBIR,  DUS&T  etc)  and  their  affect  on 
AFRL;  and  an  exploration  into  the  policy  making  on  technology  transition.  The 
qualitative  data  from  this  study,  paired  with  future  quantitative  data,  proposes  an  in  depth 
look  into  the  phenomenon  of  technology  transition. 

5.6  SUMMARY 

The  purpose  of  this  study  was  to  detennine  what  extent  AFRL  was  implementing 
known  technology  transition  practices.  It  was  found  that  the  practitioners  did  exploit 
technology  transition  practices,  but  did  not  exploit  all  of  the  best  practices  focused  on  in 
this  research.  It  was  observed  that  S&T  PMs  need  ample  help  from  specific  skill  sets  to 
efficiently  transition  technology.  As  it  was  stated  in  earlier  chapters,  to  date  there  had  not 
been  much  research  on  transitioning  AFRL  technologies.  This  study  proposes  action  for 
future  research  in  technology  transition.  Information  acquired  through  the  literature 
review  and  interviews  accelerates  the  process  toward  gaining  a  better  understanding  of 
technology  transition. 
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